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The  Research  and  Technology 
Organisation  (RTO)  of  NATO 


RTO  is  the  single  focus  in  NATO  for  Defence  Research  and  Technology  activities.  Its  mission  is  to  conduct  and  promote 
co-operative  research  and  information  exchange.  The  objective  is  to  support  the  development  and  effective  use  of 
national  defence  research  and  technology  and  to  meet  the  military  needs  of  the  Alliance,  to  maintain  a  technological 
lead,  and  to  provide  advice  to  NATO  and  national  decision  makers.  The  RTO  performs  its  mission  with  the  support  of  an 
extensive  network  of  national  experts.  It  also  ensures  effective  co-ordination  with  other  NATO  bodies  involved  in  R&T 
activities. 


RTO  reports  both  to  the  Military  Committee  of  NATO  and  to  the  Conference  of  National  Armament  Directors. 
It  comprises  a  Research  and  Technology  Board  (RTB)  as  the  highest  level  of  national  representation  and  the  Research 
and  Technology  Agency  (RTA),  a  dedicated  staff  with  its  headquarters  in  Neuilly,  near  Paris,  France.  In  order  to 
facilitate  contacts  with  the  military  users  and  other  NATO  activities,  a  small  part  of  the  RTA  staff  is  located  in  NATO 
Headquarters  in  Brussels.  The  Brussels  staff  also  co-ordinates  RTO’s  co-operation  with  nations  in  Middle  and  Eastern 
Europe,  to  which  RTO  attaches  particular  importance  especially  as  working  together  in  the  field  of  research  is  one  of  the 
more  promising  areas  of  co-operation. 


The  total  spectrum  of  R&T  activities  is  covered  by  the  following  7  bodies: 


•  AVT 

•  HFM 

•  1ST 

•  NMSG 

•  SAS 

•  SCI 

•  SET 


Applied  Vehicle  Technology  Panel 
Human  Factors  and  Medicine  Panel 
Information  Systems  Technology  Panel 
NATO  Modelling  and  Simulation  Group 
System  Analysis  and  Studies  Panel 
Systems  Concepts  and  Integration  Panel 
Sensors  and  Electronics  Technology  Panel 


These  bodies  are  made  up  of  national  representatives  as  well  as  generally  recognised  ‘world  class’  scientists.  They  also 
provide  a  communication  link  to  military  users  and  other  NATO  bodies.  RTO’s  scientific  and  technological  work  is 
carried  out  by  Technical  Teams,  created  for  specific  activities  and  with  a  specific  duration.  Such  Technical  Teams  can 
organise  workshops,  symposia,  field  trials,  lecture  series  and  training  courses.  An  important  function  of  these  Technical 
Teams  is  to  ensure  the  continuity  of  the  expert  networks. 

RTO  builds  upon  earlier  co-operation  in  defence  research  and  technology  as  set-up  under  the  Advisory  Group  for 
Aerospace  Research  and  Development  (AGARD)  and  the  Defence  Research  Group  (DRG).  AGARD  and  the  DRG  share 
common  roots  in  that  they  were  both  established  at  the  initiative  of  Dr  Theodore  von  Karman,  a  leading  aerospace 
scientist,  who  early  on  recognised  the  importance  of  scientific  support  for  the  Allied  Armed  Forces.  RTO  is  capitalising 
on  these  common  roots  in  order  to  provide  the  Alliance  and  the  NATO  nations  with  a  strong  scientific  and  technological 
basis  that  will  guarantee  a  solid  base  for  the  future. 


The  content  of  this  publication  has  been  reproduced 
directly  from  material  supplied  by  RTO  or  the  authors. 
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Virtual  Institute  on  Human 
Behavior  Representation 
(RTO-TR-SAS-053) 


Executive  Summary 


The  topic  and  focus  of  the  SAS-053  was  extended  and  extrapolated  from  a  previous  NATO  Long  Term 
Scientific  Study  (LTSS/51)  on  Human  Behavior  Representation  sponsored  by  the  SAS  Panel.  The  purpose 
of  the  LTSS  was  to  provide  a  report  for  the  use  by  NATO  and  national  authorities  on  the  implications  of 
technological  developments  to  military  operations  over  the  next  ten  to  fifteen  years  and  to  provide 
research  planners  with  recommendations  for  research  that  address/investigate  the  implications  of  projected 
technological  developments.  Of  particular  interest  to  this  project,  under  LTSS/51,  their  recommendation  of 
highest  priority  was  to  establish  a  NATO  RTO  exploratory  team  to  investigate  the  feasibility  and  utility  of 
assembling  a  virtual  institute  for  research  on  human  behavior  modeling  (NATO,  2001).  Specifically, 
the  virtual  institute  was  to  be  established  for  the  purpose  of  modeling  military-related  individuals,  teams, 
groups,  platforms,  and  organizations  in  their  performance  of  military  operations  and  tasks.  The  virtual 
institute  was  envisioned  as  a  NATO  resource  that  would  accelerate  technology  dissemination  in  specific 
technology  areas  that  would  include  development  of  a  web-based  clearing  house  of  databases,  models  and 
model  components  and  developing  standards,  and  requirements  to  support  multi-national  research  on 
human  behavior  modeling. 

The  objective  of  SAS-053  was  two-fold.  Firstly,  SAS-053  is  to  advise  the  SAS  Panel  and,  through 
cooperation  with  other  RTO  Panels,  the  RTO  on  the  technical  merit  and  feasibility  of  establishing  a  virtual 
institute  within  NATO.  Secondly,  the  SAS-053  is  to  advise  the  SAS  Panel  and,  through  cooperation  with 
other  RTO  Panels,  the  RTO  on  the  feasibility  of  enabling  multi-national  research  and  development  in 
human  behavior  modeling  by  using  the  virtual  institute  concept.  The  SAS-053  is  able  to  present  three 
findings.  The  first  finding  is  that  the  virtual  institute  concept  has  sound  technical  merit  and  its 
technological  foundations  are  growing  stronger  due  to  commercial  sector  technology  developments.  The 
second  finding  is  that  the  virtual  institute  concept  is  viable,  feasible,  and  shows  great  promise  for 
providing  research  cost  savings  and  improved  international  research  collaboration.  The  third  finding  is  that 
the  virtual  institute  concept  is  an  advisable  means  for  conducting  international,  multi-cultural  research  in 
human  behavior  modeling. 

As  a  first  step,  meetings  were  held  to  discuss  technical  feasibility  of  the  concepts,  to  conduct  technical 
interchange  and  to  assess  NATO-wide  support  for  the  concepts.  In  addition,  team  members  participated  in 
their  individual  countries’  technical  activities  related  to  the  two  concepts  in  order  to  assess  their  own 
countries’  technical  experts’  opinions  concerning  the  concepts  as  well  as  support  for  the  concepts  and 
commitment  to  their  execution.  A  final  thrust  was  to  organize  and  prepare  to  assess  developments  of  the 
technological  foundations  needed  to  implement  and  operate  a  virtual  institute. 

In  brief,  there  is  support  for  the  concepts;  the  technologies  needed  to  implement  the  concepts  are  available 
and  have  been,  in  many  cases,  demonstrated;  and  there  is  widespread  belief  that  the  NATO  community 
would  benefit  from  the  virtual  institute  concept  and  that  human  behavior  representation  is  one  of  many 
research  arenas  that  would  benefit  by  exploiting  the  virtual  institute.  There  are  a  number  of  promising 
follow-on  projects  that  a  follow-on  Panel  should  pursue  in  the  areas  of  human  behavior  modeling  research 
and  uses  of  the  virtual  institute.  A  key  activity  for  the  virtual  institute  should  be  the  development  of 
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improved  human  behavior  models.  Improvement  in  human  behavior  models  requires  research  on  the 
incorporation  of  multiple  skill  levels  and  representations  of  doctrine  and  they  must  reflect  societal, 
training,  and  educational  differences.  Behavior  moderators  such  as  morale,  fatigue,  stress,  fear, 
information  operations,  and  intent  must  be  modeled  and  included  within  the  human  behavior  model.  We 
also  believe  that  these  projects  can  be  given  focus  and  provide  demonstrable  utility  by  producing  a  tool 
that  could  be  used  among  the  nations;  our  suggestion  is  the  development  of  a  multi-national,  coalition 
submarine  warfare  simulation  environment.  This  environment  would  provide  opportunities  for  cultural 
and  team  research,  modeling  and  simulation  within  each  submarine  as  well  as  for  cross-cultural  and  multi¬ 
national  modeling  and  simulation  in  the  command  and  control  of  submarine  operations. 
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Institut  virtuel  de  representation 
du  comportement  humain 

(RTO-TR-SAS-053) 


Synthese 

Le  sujet  et  le  cadre  du  SAS-053  ont  ete  elargis  et  extrapoles  a  partir  d’une  precedente  etude  scientifique  a 
long  terme  (LTSS/51)  de  l’OTAN,  consacree  a  la  representation  du  comportement  humain  et  sponsorisee  par 
la  commission  SAS.  Cette  LTSS  avait  pour  objet  de  soumettre  un  rapport  a  destination  de  l’OTAN  et  des 
autorites  nationales,  portant  sur  les  implications  des  developpements  technologiques  pour  les  operations 
militaires  dans  les  dix  a  quinze  prochaines  annees,  et  de  foumir  aux  responsables  de  la  planification  des 
recherches  des  recommandations  pour  les  recherches  abordant/etudiant  les  implications  des  developpements 
technologiques  prevus.  D’un  interet  tout  particulier  pour  ce  projet,  la  recommandation  la  plus  prioritaire  de  la 
LTSS/51  concemait  la  mise  en  place  d’un  groupe  d’etude  RTO  OTAN  charge  d’analyser  la  faisabilite  et 
1’utilite  de  la  creation  d’un  institut  virtuel  de  recherche  sur  la  modelisation  du  comportement  humain 
(OTAN,  2001).  Plus  specifiquement,  cet  institut  virtuel  devait  avoir  pour  vocation  de  modeliser 
l’accomplissement  de  taches  et  d’ operations  militaires  par  des  personnes,  equipes,  groupes,  plates-formes  et 
organisations  en  rapport  avec  l’armee.  L’institut  virtuel  etait  envisage  comme  une  ressource  de  l’OTAN 
permettant  d’accelerer  la  diffusion  technologique  dans  des  domaines  technologiques  precis,  et  comprenait  le 
developpement  d’un  centre  d’echange  en  ligne  de  bases  de  donnees,  de  modeles,  de  normes  de 
developpement  et  elements  de  modeles,  et  des  besoins  relatifs  aux  recherches  multinationales  sur  la 
modelisation  du  comportement  humain. 

L’objectif  du  SAS-053  etait  double.  Premierement,  le  SAS-053  devait  conseiller  la  commission  SAS  - 
et,  par  le  biais  de  sa  cooperation  avec  d’autres  commissions,  la  RTO  -  sur  la  faisabilite  et  1’ interet  technique 
de  la  creation  d’un  institut  virtuel  au  sein  de  l’OTAN.  Deuxiemement,  le  SAS-053  devait  conseiller  la 
commission  SAS  -  et,  par  le  biais  de  sa  cooperation  avec  d’autres  commissions,  la  RTO  -  sur  la  possibility 
de  permettre  la  recherche  et  le  developpement  multinationaux  en  matiere  de  modelisation  du  comportement 
humain  grace  a  l’emploi  du  concept  d’institut  virtuel.  Le  SAS-053  est  en  mesure  de  presenter  trois 
conclusions.  La  premiere  conclusion  est  que  le  concept  d’institut  virtuel  represente  un  interet  technique 
certain  et  que  les  progres  de  la  technologie  du  secteur  commercial  renforcent  en  permanence  ses  fondements 
technologiques.  La  deuxieme  conclusion  est  que  le  concept  d’institut  virtuel  est  viable,  realisable  et 
egalement  tres  prometteur  quant  a  la  realisation  d’ economies  et  1’ amelioration  de  la  collaboration 
intemationale  dans  le  domaine  de  la  recherche.  La  troisieme  conclusion  est  que  le  concept  d’institut  virtuel 
est  un  procede  que  l’on  peut  recommander  pour  mener  des  recherches  intemationales  et  multiculturelles  sur 
la  modelisation  du  comportement  humain. 

Dans  un  premier  temps,  plusieurs  reunions  ont  ete  tenues  en  vue  d’etudier  la  faisabilite  technique  des 
concepts,  de  proceder  a  des  echanges  techniques  et  d’evaluer  le  soutien  recueilli  par  ces  concepts  au 
niveau  de  l’OTAN.  En  outre,  certains  membres  du  groupe  ont  participe  aux  activites  techniques  de  leur 
pays  relatives  aux  deux  concepts  afin  d’evaluer  les  avis  de  leurs  «  experts  techniques  »  nationaux  sur  ces 
questions,  ainsi  que  le  soutien  recueilli  par  ces  concepts  et  les  engagements  pris  pour  leur  mise  en  oeuvre. 
La  derniere  initiative  a  consiste  a  organiser  et  preparer  T  evaluation  des  developpements  des  fondements 
technologiques  necessaires  a  la  mise  en  oeuvre  et  a  1’ exploitation  d’un  institut  virtuel. 

En  resume,  ces  concepts  trouvent  un  certain  soutien  ;  les  technologies  requises  pour  leur  mise  en  oeuvre 
existent  et  ont  ete  eprouvees  dans  de  nombreux  cas  ;  et  il  est  largement  admis  que  la  communaute  de 


RTO-TR-SAS-053 


ES-3 


NATO 

OTAN 


ORGANIZATION 


l’OTAN  profitera  de  ce  concept  d’institut  virtuel  et  que  la  representation  du  comportement  humain  est  l’un 
des  nombreux  domaines  de  recherche  qui  beneficieraient  de  l’exploitation  de  l’institut  virtuel.  Un  certain 
nombre  de  projets  inherents  prometteurs  devraient  etre  approfondis  par  une  commission  de  suivi  dans  les 
domaines  de  la  recherche  sur  la  modelisation  du  comportement  humain  et  des  utilisations  de  l’institut  virtuel. 
L’une  des  activites  essentielles  de  l’institut  virtuel  devrait  etre  le  developpement  de  modeles  de 
comportement  humain  plus  perfectionnes.  L’ amelioration  des  modeles  de  comportement  humain  necessite 
des  recherches  sur  1’ incorporation  de  plusieurs  niveaux  de  competences  et  de  representations  de  doctrines  ; 
ces  modeles  doivent  egalement  refleter  les  disparites  liees  a  la  societe,  la  formation  et  1’ education. 
Les  moderateurs  de  comportement  tels  que  le  moral,  la  fatigue,  le  stress,  la  peur,  les  operations  cognitives  et 
l’intention  doivent  etre  modelises  et  integres  dans  le  modele  de  comportement  humain.  Nous  pensons 
egalement  que  ces  projets  peuvent  faire  l’objet  d’une  attention  particuliere  et  s’averer  utiles  en  foumissant  un 
outil  que  les  nations  pourront  employer ;  nous  suggerons  le  developpement  d’un  environnement  de 
simulation  multinational  de  guerre  sous-marine  de  la  coalition.  Un  tel  environnement  favorisera  la 
modelisation,  la  simulation  et  les  recherches  culturelles  et  par  equipes  au  sein  de  chaque  sous-marin,  ainsi 
que  la  modelisation  et  la  simulation  interculturelles  et  multinationales  lors  du  commandement  et  du  controle 
des  operations  sous-marines. 
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Background 

•  LTSS  SAS-017  on  Human  Behaviour 
Representation  (HBR)  Recommendations 

-  Initiate  an  Exploratory  Team  (ET)  to  build  up  a 
Virtual  Institute  for  Research  on  HBR 

-  Initiate  an  Exploratory  Team  (ET)  on  a  Research 
Plan  for  Team,  Group  and  Organizational  Modelling 
Research 

•  SASET.V 

Virtual  Institute  for  Research  on  HBR 

-  Combined  Two  Recommended  Activities 

-  Recommendation  to  Establish  an  RTG  for  a  Virtual 
Institute  with  a  Specific  Task  as  Test  Case  (HBR) 
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Scope  and  Goals 

•  Develop  a  Virtual  Institute  Concept  for  NATO 

-  Organizational 

-  Technical 

-  Multi-national  Use  and  Involvement 

-  Virtual  Institute  Metrics  and  Success  Criteria 

•  Develop  a  Roadmap  for  the  Development  and 
Use  of  Virtual  Institutes  (VI)  in  NATO 

•  Employ  Prototype  VI 

-  Assess  Utility  of  VI  Concept 

-  Research  on  Team,  Group,  Organization  HBR 
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■  •  USA 
NC3A 


Canada 
Germany 
The  Netherlands 
United  Kingdom 
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Meetings  (1  of  3) 

•  1st  Meeting:  September  2004  -  NATO  HQ 

-  Virtual  Institute  Framework 

-  State-of-the-Art  for  Team,  Group,  and 
Organisational  Modelling 

•  2nd  Meeting:  February  2005  Orlando,  USA 
(with  SAS-050) 

-  Co-Meeting  with  SAS-050 

-  Experiment  with  HFM-Panel/ACT  Co-operation 
(HFM-138  Adaptability  in  Coalition  Teamwork) 
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Meetings  (2  of  3) 

•  3rd  Meeting  October  2005  -  NC3A,  The  Hague 

-  Progress  on  State-of-the-Art  Report  on  Team, 
Group,  and  Organisational  Modelling  (DEU  Input) 

-  Progress  on  VI/HBR  Concept  of  Operation 

-  VI/HBR  Prototype  Experiment 

•  4th  Meeting  May  2006  -  IDA,  Alexandria,  USA 

-  Study  on  Team  Oriented  Behaviour  in  MNE  4 

-  State-of-the-Art  Report 

-  Comparison  of  HBR  Approaches 

-  Experiment  LTAMC  (NATO  Leader  and  Team 
Adaptability  in  Multi-national  Coalitions) 

23  October  2007 


Meetings  (3  of  3) 

•  5th  Meeting  June  2007  -  IABG,  Ottobrunn, 
Germany 

-  HFM-138  Update 

-  Layout  of  Final  Report 

-  Discussion  on  Results  Lessons  Identified 

-  Discussion  on  Recommendations 
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input  to  Other  Activities 

•  Meetings  with  NATO  and  Nations 

-  Adm  Bartoli  on  Support  for  VI 

-  ACT 

-  US  JFCOM 

•  Contacts  with  the  HBR  Community 

-  BRIMS  (Behavior  Representation  in  Modeling  And 
Simulation)  Presentations 

-  HFM  Panel 

-  Other  Conferences 

•  RTA  VI  Meeting 
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Final  Report 

•  Executive  Summary 

•  Introduction 

•  SAS-053  Work 

•  State  of  the  Art  HBR  Team,  Group, 
Organization 

•  Virtual  Institute  Concept 

•  Results  and  Lessons  Identified 

•  Recommendations 
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HBR  State-of-the-Art  Report 

•  Introduction 

•  Modeling  Parameters  in  Team,  Group,  and 
Organizational  Modeling 

-Preliminary  Ontology  Development  Necessary  for 
Recommended  HFM  RTG 

•  Usage  and  Applications 

-Major  Applications  under  Development  in  Nations 
Relevant  to  Technology  Review 

-Incorporation  of  Commercial  Gaming  Technology 
into  Modeling  Review 

•  Shortfalls  in  Modeling  Applications  for  Team, 
Group,  and  Organizational  Modeling 
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s's|,fel  Virtual  Institute  Concept 

•  Introduction 

•  Definition 

•  Underlying  Technology 

•  Organisation  of  an  Virtual  Institute 

•  Usage 

•  Data  Exchange 

•  Current  Projects 
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Developments  in  IT/VI  Technology 

•  Study  was  Recommended  in  1999  and  Started 
in  2004  -  Now  we  have  2007! 

•  IT  Technology  now  Commercially  Available 
for  Everybody  (High  Bandwidth,  Video, 
Groupware,  Shared  Workspaces) 

•  Web  2.0  Developments  (.Mac,  MySpace, 
YouTube,  Google,  BlogSpot,  Yahoo, 

Wikipedia, ...) 

•  But  Still  Somebody  has  to  Take  Responsibility 
for  Organization,  Version  Control, ... 
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Data  Exchange 

•  Security 

-  Cost  Factor!  (e.g.  CFBL) 

•  Data  Exchange  Agreements 

-  NATO  Policy  Necessary 

—  Multinational  (e.g.  MNE  4) 

-  National 

•  Intellectual  Property 

-  Commercial  Interests 
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SA-053  Results 

•  Support  HFM-1 38 

-  Experiment  Setup 

•  Support  SAS-050 

-  Discussion  of  Follow  on  Work 

•  Input  to  RTA  VI  Setup 

-  Use  of  WISE  Portal 

-  Input  to  VI  Meeting 

•  Input  to  RTB  Strategic  Plan 
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Lessons  Identified 

•  Study  Team 

-Commitment  of  People  is  not  Enough  -- 
Budget  must  be  Provided  Too 

-Change  of  Team  Members  Cost  Time 

•  Dealing  with  Fast  Moving  Technology 
-RTB  Procedures  don’t  Fit  for  Rapid  Development 
-Budgeting  Process  in  NATO  doesn’t  Fit  Either 

•  Provision/Share  of  Knowledge  Critical 
(Data  Exchange  Agreements,  Classification) 

NATO  Not  Ready  for  VI  yet  ? 


23  October  2007 


Recommendations 

•  SAS  RSG  -  Setup  and  Administration  of  HBR 
VI  in  cooperation  with  RTA  IS 

•  HFM  RSG  -  HBR  Team  and  Organization 
Ontology  Development 

•  HFM  RSG  -  Multinational  Experiment  HFM- 
138  Follow  On 

•  HFM/SAS  RSG  -  Adopt  Exisiting  Technology 
in  Developing  Podcast  Lecture  Series  for  RTA 

•  SAS  RSG  -  Data  Exchange  Practises  in  NATO 

•  HFM  RSG  -  The  Use  of  Virtual  Worlds  for 
NATO  R&D  in  HBR 
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Chapter  1  -  INTRODUCTION 


Research  and  Technology  (R&T)  plays  a  major  role  in  NATO  in  building  capabilities  that  are  required  to 
fulfill  NATO’s  mission.  In  the  past  the  main  objective  of  R&T  was  to  build  more  effective  weapon  systems 
and  to  enhance  command,  control  and  communications. 

Since  the  end  of  the  cold  war  experience  has  shown  that  conflict  resolution  requires  the  application  of  all 
elements  of  national  and  international  power  -  political,  diplomatic,  economic,  financial,  informational, 
social,  and  commercial,  as  well  as  military.  To  resolve  conflicts  or  crises,  the  North  Atlantic  Treaty 
Organisation  (NATO)  adopted  a  Comprehensive  Approach  that  enables  the  collaborative  engagement  of 
all  requisite  civil  and  military  elements  of  international  power  to  end  hostilities,  restore  order,  commence 
reconstruction,  and  begin  to  address  a  conflict’s  root  causes.  NATO  provides  the  military  element  for  a 
comprehensive  approach.  Many  other  national,  international,  and  non-governmental  actors  can  provide  the 
civilian  elements.  In  this  environment  the  focus  of  R&T  has  to  change  to  include  more  “soft”  sciences  like 
social  sciences,  economics,  psychology,  etc.  Especially  Human  Behaviour  Representation  (HBR)  plays  an 
important  role  in  today’s  scenarios  with  opposing,  neutral  and  friendly  actors  in  a  given  conflict. 

This  change  has  a  dramatic  impact  on  the  means  to  conduct  R&T.  In  the  past  conferences,  workshops  and 
meetings  served  well  for  the  research  community  and  the  decision  makers  to  exchange  information, 
to  publish  research  reports  and  to  remain  appraised  of  relevant  scientific  developments.  Technology 
development  has  now  a  much  higher  rate  of  change  and  capabilities  are  needed  that  support  an  effective 
and  efficient  collaboration  of  geographically  distributed  research  teams  and  the  dissemination  of  research 
results  and  other  relevant  expertise.  With  the  Comprehensive  Approach  the  research  community  has 
grown  and  in  many  cases  multi-disciplinary  research  is  the  only  way  to  achieve  the  desired  results. 

Such  a  capability  needed  could  be  a  Virtual  Institute  (VI).  “Virtual”  is  defined  in  the  Oxford  Concise 
Dictionary  as  “not  physically  existing  as  such  but  made  [by  software]  to  appear  to  do  so”.  “Institute”  is 
defined  as  “a  society  or  organization  for  the  promotion  of  science,  education,  etc.”.  Given  both  definitions 
a  Virtual  Institute  could  be  seen  as  a  “non-physical,  but  nevertheless  fully  functioning  and  capable 
organism  promoting  the  development  and  use  of  research  and  technology”. 

The  technology  to  implement  such  a  Virtual  Institute  is  commercially  available  and  has  only  recently 
become  available  at  a  cost  and  performance  that  makes  their  use  efficient  and  effective  for  all  NATO 
members.  The  more  challenging  part  of  building  a  Virtual  Institute  in  a  multi-national  environment  lies  in 
the  build-up  of  the  organization  of  such  an  entity  and  the  procedures  how  to  operate. 

1.1  SHORT  HISTORY 

The  NATO  Long  Term  Scientific  Study  (LTSS/51)  on  Human  Behaviour  Representation  sponsored  by  the 
SAS  Panel  published  a  report  for  the  use  by  NATO  and  national  authorities  on  the  implications  of 
technological  developments  to  military  operations  over  the  next  ten  to  fifteen  years.  An  important  part  of 
the  report  is  recommendations  for  research  that  address/investigate  the  implications  of  projected 
technological  developments.  The  recommendation  of  highest  priority  was  to  establish  a  NATO  RTO  study 
to  investigate  the  feasibility  and  utility  of  assembling  a  Virtual  Institute  for  research  on  human  behaviour 
modelling  (NATO,  2001).  The  Virtual  Institute  was  envisioned  as  a  NATO  resource  that  would  accelerate 
technology  dissemination  in  specific  technology  areas  that  would  include  development  of  a  web-based 
clearing  house  of  databases,  models  and  model  components  and  developing  standards,  and  requirements  to 
support  multi-national  research  on  human  behaviour  modelling. 

This  particular  LTSS/51  recommendation  was  presented  to  and  approved  by  the  SAS  Panel,  which  in  turn 
established  the  SAS-ET.V  exploratory  team.  The  objective  of  the  exploratory  team  was  two-fold.  Firstly, 


1  -1 


RTO-TR-SAS-053 


INTRODUCTION 


it  had  to  advice  the  SAS  Panel  and,  through  cooperation  with  other  RTO  Panels,  the  RTO  on  the  technical 
merit  and  feasibility  of  establishing  a  Virtual  Institute  within  NATO.  Secondly,  the  SAS-ET.V  had  to  advice 
the  SAS  Panel  and,  through  cooperation  with  other  RTO  Panels,  the  RTO  the  feasibility  of  enabling  multi¬ 
national  research  and  development  in  human  behaviour  modelling  by  using  the  virtual  institute  concept. 

USA  (Lead  Nation),  Canada,  Germany,  Netherlands,  United  Kingdom,  and  NATO  C3  Agency  were 
participating  in  the  exploratory  team.  To  address  the  two  objectives  for  the  Panel,  several  different  activities 
were  undertaken  to  assess  the  feasibility  and  desirability  of  the  two  concepts.  Three  meetings  were  held  to 
discuss  the  technical  feasibility  of  the  concepts,  to  conduct  technical  interchange  and  to  assess  NATO-wide 
support  for  the  concepts.  In  addition,  team  members  participated  in  their  individual  countries’  technical 
activities  related  to  the  two  concepts  in  order  to  assess  their  own  countries’  technical  experts’  opinions 
concerning  the  concepts  as  well  as  support  for  the  concepts  and  commitment  to  their  execution.  A  final  area 
of  endeavour  was  an  ongoing  assessment  of  the  technological  foundations  needed  to  implement  and  operate 
a  virtual  institute.  In  brief,  the  team  found  support  for  the  concepts;  the  technologies  needed  to  implement  the 
concepts  are  available  and  have  been,  in  many  cases,  demonstrated;  and  there  is  widespread  belief  that  the 
NATO  community  would  benefit  from  the  virtual  institute  concept  and  that  human  behaviour  representation 
is  an  appropriate  choice  for  the  demonstration  of  the  virtual  institute  concept. 

The  exploratory  team  briefed  the  SAS  Panel,  which  decided  to  approve  the  Research  Task  Group  (RTG) 
activity  SAS-053  “Virtual  Institute  on  Human  Behaviour  Representation”.  The  scope  and  goals  for  the 
study  are: 

•  Develop  a  Virtual  Institute  Concept  for  NATO: 

•  Organizational; 

•  Technical; 

•  Multi-national  Use  and  Involvement;  and 

•  Virtual  Institute  Metrics  and  Success  Criteria. 

•  Develop  a  Roadmap  for  the  Development  and  Use  of  Virtual  Institutes  (VI)  in  NATO. 

•  Employ  Prototype  VI  on  HBR: 

•  Assess  Utility  of  VI  Concept;  and 

•  Research  on  Team,  Group,  Organization  HBR. 

The  following  nations  participated  in  the  study: 

•  USA  (Co-Lead); 

•  NC3A  (Co-Lead); 

•  Canada; 

•  Germany; 

•  Netherlands;  and 

•  United  Kingdom. 


1.2  OUTLINE  OF  THE  REPORT 

The  Executive  Summary  gives  an  overview  on  the  study  and  highlights  the  most  important  findings  of  the 
study  for  the  decision  makers.  The  introduction  shows  why  the  topic  “Virtual  Institute”  is  important  for 
NATO  and  what  can  be  supported  in  the  Research  and  Technology  area  with  the  Virtual  Institute  concept. 
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It  also  highlights  the  history,  which  led  to  the  study  activity.  A  short  overview  on  the  report  concludes  the 
introduction.  Chapter  2  summarizes  the  SAS-053  work,  gives  an  overview  on  the  findings  of  the  study 
group  meetings,  highlights  the  results  of  meetings  with  NATO  and  nations  regarding  the  build  up  of  a  VI, 
shows  the  contacts  with  the  HBR  community  and  summarizes  a  meeting  at  RTA  regarding  VI.  Chapter  3 
consists  of  the  State  of  the  Art  report  on  team,  group  and  organizational  modelling.  After  a  short 
introduction  the  modelling  parameters  in  team,  group  and  organizational  modelling  are  described.  Usage 
and  Applications  is  followed  by  an  analysis  on  shortfalls.  Chapter  4  shows  the  VI  Concept.  After  an 
introduction  and  the  definition  of  a  VI  the  underlying  technology  will  be  reviewed.  The  organization  of  a 
VI  is  developed  and  discussed.  Usage  and  data  exchange  in  a  multi-national  environment  are  examined. 
The  chapter  closes  with  an  overview  on  relevant  current  projects.  Chapter  5  summarizes  the  results  of  the 
study  and  lessons  identified.  The  report  closes  with  Chapter  6  on  recommendations  to  the  SAS  Panel 
regarding  follow-on  studies. 
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Chapter  2  -  SAS-053  WORK 

Main  instrument  in  the  work  of  the  SAS-053  study  group  were  the  study  group  meetings.  These  meetings 
were  used  to  first  initiate  the  work  and  to  discuss  with  all  members  the  contents  of  the  study  and  later  to 
present  and  discuss  inputs  to  the  study  by  the  group  members  and  to  coordinate  the  work.  Beside  the 
meetings  of  the  study  group  the  co-chairmen  and  study  team  members  had  meetings  with  NATO  and 
nations.  These  meetings  were  intended  to  introduce  decision  makers  to  the  work  of  the  study  group  and  to 
ask  for  financial  support  for  the  build-up  of  the  Virtual  Institute  for  Human  Behaviour  Representation. 
A  third  area  of  work  was  the  connection  to  the  Human  Behaviour  Representation  Community.  BRIMS 
(Behavior  Representation  in  Modelling  and  Simulation)  and  other  conferences  were  used  to  present  the 
work  of  the  study  group  and  to  get  feedback  from  the  community.  Also  a  close  connection  to  the  Human 
Factors  and  Medicine  (HFM)  Panel  was  established.  A  special  meeting  of  the  Research  and  Technology 
Agency  (RTA)  on  VI  was  supported. 


2.1  SAS-053  STUDY  GROUP  MEETINGS 

Five  study  group  meetings  were  held: 

•  1st  Meeting  September  2004  at  NATO  HQ,  BEL; 

•  2nd  Meeting  February  2005  in  Orlando,  USA  (co-meeting  with  SAS-050); 

•  3rd  Meeting  October  2005  at  NC3A,  The  Hague,  NLD; 

•  4th  Meeting  May  2006  at  IDA,  Alexandria,  USA;  and 

•  5th  Meeting  June  2007  at  IABG,  Ottobrunn,  DEU. 

2.1.1  1st  Meeting  September  2004  -  NATO  HQ,  BEL 

The  first  meeting  started  with  the  initiation  of  the  study.  Mr.  Cornelis  Bouman  (SAS  Panel  Executive) 
provided  an  introduction  in  RTO  and  its  study  work.  The  participants  introduced  themselves  and  gave  a 
short  overview  on  the  interest  of  their  nations  in  the  study  area  and  related  work  underway. 

The  chairman  of  the  ET.V,  Dr.  Sheila  Banks,  presented  the  ET.V  report,  the  proposed  Terms  of  Reference 
(TOR)  and  the  Technical  Activity  Proposal  (TAP),  which  were  approved  by  the  group. 

The  study  group  discussed  links  to  other  RTO  study  groups.  SAS-050  “Exploring  new  C2  Concepts  and 
Capabilities”,  SAS-057  “Information  Operations”  and  the  Long  Term  Scientific  Study  (LTSS)  on  Non- 
Lethal  Weapons  were  identified.  Also  a  link  to  the  HFM  Panel  was  seen  to  be  necessary. 

Phil  Reed,  the  head  of  Information  Management  and  Systems  Branch  of  RTA  gave  a  presentation  of  RTO 
Information  Management  and  the  NATO  Technology  Taxonomy  Database.  The  RTO  WISE  Information 
Portal  was  in  the  status  of  being  transferred  from  ACT  to  an  own  RTA  server.  SAS-053  decided  to  use  it 
as  a  starting  point  for  the  VI. 

As  a  start  for  the  work  in  the  HBR  area  the  State-of-the-Art  for  Team,  Group,  and  Organisational  Modelling 
was  discussed. 

Main  action  items  of  this  meeting  were  establishing  links  to  the  identified  other  study  groups,  the  build  of 
an  SAS-053  WISE  portal,  draft  of  an  VI  concept,  proposals  for  an  experiment  using  the  VI  and  input  to 
the  work  on  the  state-of-the-art  report. 


RTO-TR-SAS-053 


2-1 


SAS-053  WORK 


2.1.2  2nd  Meeting  February  2005  -  Orlando,  USA  (with  SAS-050) 

The  2nd  meeting  was  a  co-meeting  with  the  SAS-050  study  group.  Both  groups  presented  their  studies  and 
discussed  a  possible  collaboration.  Because  of  the  timelines  of  both  studies  SAS-050  couldn’t  participate 
directly  with  an  experiment  in  SAS-053  but  stated  that  they  are  planning  for  a  follow  on  activity. 
Nevertheless  cultural  variables  identified  in  SAS-050  could  be  used  for  a  setup  of  an  SAS-053  experiment. 

The  group  discussed  possible  experiments  in  the  HBR  area  and  decided  to  coordinate  with  HFM-138 
Adaptability  in  Coalition  Teamwork. 

The  group  started  with  the  preparation  of  the  planned  briefing  for  Adm.  Bartoli,  Deputy  ASG  Defence. 

2.1.3  3rd  Meeting  October  2005  -  NC3A,  The  Hague,  NLD 

The  3rd  meeting  started  with  an  update  on  the  briefing  of  Adm.  Bartoli  and  other  activities  regarding  the 
involvement  of  NATO  and  nations  in  the  build-up/funding  of  a  VI.  Everybody  is  very  interested  but 
nobody  is  able  to  fund  the  activity. 

The  WISE  portal,  which  was  used  by  the  group,  had  technical  problems  after  the  move  to  a  new  server, 
so  that  the  practical  work  for  the  VI  was  impossible. 

The  VI  concept  of  operation  was  discussed  with  the  group  and  a  first  draft  put  together. 

Nations  presented  their  input  to  the  state-of-the-art  report.  The  structure  and  content  of  the  report  was 
discussed. 

An  update  on  the  planned  experiment  together  with  HFM-138  on  understanding  cultural  diversity  in 
teamwork  was  given  and  discussed  with  the  group.  The  VI  could  be  used  in  this  experiment  during  the  set-up 
of  the  experiment  in  sharing  and  discussing  experiment  results  between  the  multi-national  participants. 

2.1.4  4th  Meeting  May  2006  -  IDA,  Alexandria,  USA 

The  announced  functionality  of  the  WISE  portal  was  still  not  available  and  some  members  still  had 
problems  to  log  in.  IMC  and  RTA  planned  to  have  a  meeting  to  discuss  issues  related  to  a  permanent 
infrastructure  to  build-up  a  VI.  The  group  discussed  this  issue  and  decided  to  be  represented  in  the 
meeting.  The  group  also  discussed  other  means  readily  available  on  the  internet  to  be  used  instead. 

The  draft  VI  concept  was  discussed  and  further  improved. 

The  State-of-the-Art  Report  was  further  discussed  and  inputs  by  the  nations  (comparison  of  HBR  models, 
work  on  ontologies)  presented.  The  topic  of  gaming  was  discussed  and  decided  to  add  a  section  to  the  report. 

The  experiment  set  up  was  discussed  and  information  on  the  used  software  regarding  installation  given. 

2.1.5  5th  Meeting  June  2007  -  IABG,  Ottobrunn,  DEU 

The  structure  and  responsibilities  for  the  Final  Report  were  discussed.  It  was  agreed  that  state-of-the-art 
report  and  the  report  on  the  VI  concept  would  be  part  of  the  Final  Report.  Both  reports  were  as  draft 
presented  and  discussed. 

A  representative  from  HFM-138  gave  an  update  on  the  experiments. 

The  group  discussed  the  topic  results  and  lessons  identified  of  the  study  and  drafted  an  input  for  the  final 
report.  Recommendations  as  outcome  of  the  study  were  proposed  and  discussed.  The  group  agreed  on  a 
prioritized  list  of  recommendations  for  the  final  report. 
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2.2  MEETINGS  WITH  NATO  AND  NATIONS 
2.2.1  Adm.  Bartoli 

The  aim  of  the  briefing  was  to  inform  Adm.  Bartoli  about  the  ongoing  RTO  SAS  Panel  Research  Task 
Group  on  a  NATO  Virtual  Institute  for  Research  on  Human  Behaviour  Representation  and  especially  about 
the  resulting  short-,  medium-  and  long-term  annual  costs  for  NATO  if  this  approach  proofs  to  be  cost- 
effective. 

The  briefing  started  to  ask:  Why  a  VI  for  NATO?  A  VI  can  serve  as  an  essential  mechanism  for 
information  exchange  and  collaboration  across  the  NATO  nations.  It  can  foster  inter  RTO  Panel  research, 
faster  research  distribution,  reduce  duplication  of  effort  and  reduce  travel  times.  It  is  part  of  the  RTO 
strategic  plan  and  its  implementation. 

The  next  question  was:  Why  a  VI  for  HBR?  Human  behaviour  plays  a  major  role  in  network  centric 
warfare  in  asymmetric  environments  and  that  is  the  environment  NATO  is  operating  in.  It  helps  internally 
in  achieving  improved  effectiveness  in  considering  cultural,  structural  and  interoperability  issues  and 
externally  in  the  context  of  Effects  Based  Approach  to  Operations  in  assessing  opponents,  neutrals  and 
friends.  Research  and  development  efforts  in  the  HBR  area  are  extremely  varied  and  must  therefore 
incorporate  multiple  significant  bodies  of  work.  Active  research  groups  exist  in  different  areas  of  the  RTO 
and  in  the  nations. 

Based  on  the  answers  to  the  first  two  questions  the  objectives  of  SAS-053  were  explained.  A  major  objective 
is  the  development  of  a  VI  concept  for  NATO  with  its  technical  and  organizational  aspects.  The  test  of  such 
a  VI  in  the  area  of  HBR  is  the  second  main  objective.  The  prototype  to  be  used  for  this  test  has  to  provide 
functionality  like  web-based  clearinghouse  with  databases,  etc.,  and  a  cooperative  environment. 

Users  of  such  a  VI  would  be  in  NATO  the  RTO  with  its  Panels  and  RTGs,  NC3A,  NURC,  NATO  HQs 
(ACT,  ACO,  JFC  HQ,  JWC,  JFTC),  in  the  nations’  military  and  research  organizations  and  in  the 
international  community  scientific  and  non-governmental  organizations. 

To  build  up  such  a  capability  NATO  infrastructure  funding  is  necessary.  Reuse  of  resources  can  help  in 
building  up  the  capability.  This  could  include  the  use  of  already  existing  information  systems  in  NATO 
esp.  in  RTO  (RTO  Forum  and  WISE  Portal),  the  NATO  Combined  Federated  Battle  Laboratories  Network 
and  the  NATO  Science,  Technology,  and  Research  Network  (STARNET).  But  even  if  free  internet 
services  are  used  a  budget  for  the  organization  and  the  operation  of  the  VI  is  required.  This  could  be  in  the 
short-term  (1-2  years)  75k  $  for  each  year  to  build  up  an  operating  VI  prototype,  in  the  medium  term 
150k$  per  year  (3-5  years)  for  the  technical  base  and  a  VI  organization  to  extend  the  basic  services  and 
in  the  long  term  for  a  full  VI  250k  $  for  the  principal  VI  organization  and  technical  base  plus  100k  $  for 
each  project  area  within  the  VI. 

The  main  result  of  this  briefing  was  that  the  upper  NATO  management  is  aware  of  initiatives  regarding 
human  behaviour  representation  and  virtual  institute  approaches.  In  the  short  term  no  additional  money 
could  be  expected  to  support  the  study.  But  in  the  long  term  hopefully  a  seed  has  been  planted  that  this 
kind  of  research  has  to  be  supported. 

2.3  CONTACTS  WITH  THE  HUMAN  BEHAVIOUR  REPRESENTATION 
COMMUNITY 

Members  of  SAS-053  held  positions  on  the  conference  committee  for  the  Behavioral  Representational  in 
Modeling  and  Simulation  (BRIMS),  which  produced  many  contacts  and  project  meetings  and  interactions. 
Members  were  also  represented  on  various  group  activities  for  HFM  and  other  NATO  RTG  activities. 
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2.4  RTA  VI  MEETING 

RTA  invited  in  June  2006  to  a  meeting  on  VI.  RTA,  MSCO,  RTA  Information  Management  and  Systems 
Branch,  an  HFM  Panel  rep  and  SAS-053  rep  were  present.  Background  for  the  meeting  was  a  RTB 
initiative  to  have  a  “one  stop  place”  for  RTA  on  the  Internet.  RTA  and  IT  briefed  on  the  status  of  the 
WISE  portal  and  future  plans.  MSCO  gave  an  overview  on  actual  NMSG  project  building  up  portals 
(Pathfinder,  SMART,  SimResource  Library).  SAS-053  gave  an  overview  on  the  findings  regarding  the 
build  up  and  organization  of  a  Virtual  Institute  and  answered  questions  from  the  audience. 

The  discussion  showed  that  some  R&T  projects  use  already  technologies,  which  are  related  to  the  VI 
concept  (e.g.  sharing  of  documents  in  a  common  workspace).  But  it  also  showed  that  the  different  projects 
are  not  coordinated  with  each  other  and  no  integrated  solution  is  used.  Especially  a  coordinating  VI 
organization  is  missing.  RTA  Information  Management  and  Systems  Branch  took  the  lead  for  organizing 
technical  workshops  and  a  high  level  R&T  Portal  meeting. 
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Chapter  3  -  STATE  OF  THE  ART  SUMMARY 


The  use  of  the  virtual  institute  in  this  initial  evaluation  can  address  a  wide  range  of  issues.  On  one  end, 
the  distribution  of  human  performance  data  necessary  for  the  development  and  use  of  human  performance 
modeling  in  many  areas  of  military  operations,  such  as  mission  rehearsal  and  mission  analysis,  is  a 
worthwhile  virtual  institute  starting  point.  Other  ideas  for  the  initial  virtual  institute  prototype  included 
current  human  performance  model  validation  in  a  multi-national  arena  and  a  large  multi-national  experiment 
encompassing  many  aspects  of  team,  group,  and  organizational  modeling.  However,  the  current  HBM  efforts 
consist  of  a  hodge-podge  of  research  aimed  at  providing  human  behavior  in  the  simulation  environment 
whether  that  environment  be  for  training,  acquisition,  command  and  control,  or  decision  aiding.  However 
developmental  appropriate  and  research  necessitated  these  effort  might  be,  they  lack  on  unifying  factor: 
a  vision.  Therefore,  SAS-053  decided  on  an  approach  to  develop  the  virtual  institute  prototype  by  using  a 
baseline  HBM  project  within  one  of  the  participating  nations  and  expand  that  HBM  baseline  to  include 
multiple  nations  and  multiple  teams  within  nations  to  develop  and  build  experimental  data  for  use  in  joint 
command  team  operations,  a  highly  stressful  and  human  intensive  environment.  This  experimental  prototype 
path  allowed  our  multi-national  research  team  to  exploit  the  institute  and  its  capabilities  to  more  thoroughly 
determine  how  culture  and  other  societal  measures  influence  joint  military  operational  performance. 
This  issue  is  an  important  research  question  with  practical  military  implications  and  applications  for  the 
results  of  the  research,  but  it  has  not  been  undertaken  to  date  because  of  the  cause  of  assembling  the  requisite 
international  team.  The  virtual  institute  is  taking  the  lead  in  addressing  this  important  issue  since  it  need  not 
bring  all  of  the  researchers  to  one  location  for  them  to  perform  their  work;  rather,  they  will  be  able  to  remain 
in  their  own  labs  and  develop  an  international  joint  experimental  plan  to  address  the  research  issue. 
The  experiment  is  an  extension  of  cultural  experiments  determined  to  be  of  most  value  to  HBR  researchers. 
This  multi-national  experiment  can  investigate  the  impact  of  culture  on  teamwork  in  both  national  and  multi¬ 
national  venues.  The  experimentation  involves  conceptual  model  development  encompassing  the  impact  of 
culture  on  joint  military  operations,  extending  experimentation  methods  to  include  process  and  outcome 
measures,  and  identifying  products  for  national  use  to  improve  leader  and  team  performance  in  multi-cultural 
environments.  A  single  experimental  design  can  be  executed  in  all  the  participating  nations  to  build  a 
baseline  of  responses  from  homogenous  military  teams. 

Human  behavior  modeling  is  the  process  of  developing  computable  representations  of  human  actions 
and/or  cognition  that  occur  in  response  to  stimuli  (both  internal  and  external  to  the  human).  Human 
behavior  models  require,  as  a  basis,  an  accurate  portrayal  of  the  modeled  users’  knowledge  and 
interactions  in  a  process  called  user  modeling.  User  modeling  can  be  effectively  utilized  to  make  explicit 
the  reasoning  about  the  purpose  of  system  adaptations  and  decisions,  to  take  into  account  user  motivation, 
and  to  present  a  unified  model  of  collaborative,  cooperative,  and  adverse  behavior.  In  addition, 
employment  of  a  user  modeling  approach  allows  the  direct  incorporation  of  knowledge  derived  from 
cognitive  task  analysis  techniques  into  the  knowledge  processing  structure  of  the  user  model.  The  main 
objective  of  user  modeling  is  to  determine  what  the  user  intends  to  do  within  an  environment.  Nuten 
defines  human  (user)  intent  as  “mental  states  that  drive  actions”  or  actions  a  user  intends  to  perform  in 
pursuit  of  a  goal  state.  Brown  states  an  approach  to  predicting  user  intent  is  to  identify  the  salient 
characteristics  of  the  domain  environment  and  specifically  determine  the  goals  a  user  is  trying  to  achieve. 
This  approach  is  based  on  the  belief  that  what  a  user  intends  to  do  in  an  environment  is  the  result  of 
environmental  events  occurring  in  the  environment,  and  by  the  goals  he/she  is  trying  to  obtain  as  a 
reaction  to  stimuli.  These  goals  can  be  explicit  or  implicit.  To  achieve  a  goal,  a  user  must  perform  certain 
actions  to  achieve  the  goal.  Goals  can  be  composed  of  multiple  actions,  with  many  pre-  and  post¬ 
conditions.  Pre-conditions  include  directly  observable  events  in  the  environment  as  well  as  indirectly 
observable  events  and  cause  a  user  to  pursue  a  goal.  Researchers  from  the  fields  of  artificial  intelligence 
(AI),  human-computer  interaction  (HCI),  psychology,  education,  and  others  have  all  investigated  ways  to 
construct,  maintain,  and  exploit  user  models,  dating  back  to  at  least  the  early  1980s.  This  infusion  from 
many  disparate  research  fields  has  allowed  user  modeling  to  focus  on  the  important  contributions  from 
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each  of  the  separate  contributing  research  fields.  For  example,  the  user  modeling  community  has  been  able 
to  reap  the  benefits  from  artificial  intelligence  researchers  by  various  knowledge  representations 
developed  by  AI  researchers  such  as  logic-based  techniques,  abductive  reasoning  techniques,  semantic 
networks,  machine  learning  techniques,  Bayesian  methods,  and  neural  networks. 

The  most  important  aspects  of  developing  a  human  behavior  model  are  knowledge  acquisition ,  which  is 
acquiring  the  information  needed  to  effectively  model  human  behavior  within  an  environment  or  situation, 
knowledge  structuring ,  which  is  structuring  the  knowledge  base  into  a  form  that  can  be  accessed  rapidly 
and  used  for  analysis  and  decision-making,  and  building  the  decision-making  apparatus  to  perform 
decision  (reasoning)  in  order  to  simulate  human  behavior.  Knowledge  acquisition  and  knowledge 
structuring  jointly  determine  the  information  about  the  human  mental  models  that  is  brought  to  the 
decision  process  and  the  key  static  and  dynamic  informational  factors  in  this  process.  The  decision-making 
apparatus  is  used  to  support  both  near-term,  reactive  decision-making  and  long  term  planning  decision¬ 
making.  Different  artificial  intelligence  techniques,  such  as  case-based  reasoning,  frames,  or  fuzzy  logic, 
are  typically  used  in  either  near-term  or  long-term  decision  making  to  achieve  the  proper  behavior.  These 
three  components,  knowledge  acquisition,  knowledge  structuring,  and  decision-making  apparatus,  are  all  the 
subject  of  human  behavior  modeling  experiments.  To  be  useful,  a  human  behavior  model  must  perform  the 
following: 

1)  Correctly  compute  outputs  of  the  human  decision  making  process  at  human-scale  rate  of  time; 

2)  Be  unpredictable;  and 

3)  Be  certifiable. 

The  first  component,  correct  outputs  modeling,  forces  the  HBM  to  react  at  a  human-scale  rate  of  time  and 
requires  that  the  decisions  appear  to  be  made  by  a  human.  That  is,  random,  disjointed  sequences  of 
decisions,  decisions  that  seem  to  be  made  irregardless  of  the  world  state,  and  decisions  that  seem  to  be 
made  in  total  disregard  for  the  success  of  the  mission  should  not  occur.  The  second  component  of  human 
behavior,  unpredictability,  addresses  the  observation  that  humans  tend  to  behave  in  a  manner  such  that 
human  opponents  cannot  detect  patterns  in  behavior.  However,  unpredictability  does  not  mean  that  the 
behavior  is  random.  Rather,  this  requirement  means  that  the  behavior  should  be  as  rich  and  varied  as  a 
human’s  behavior.  The  third  component  of  human  behavior  modeling  is  that  the  behavior  be  certifiable. 
To  be  certified,  a  software  system  must  be  able  to  be  measured  against  and  compared  to  the  exhibited 
behaviors  of  a  human  in  a  comparable  situation.  Certifiability  does  not  mean  that  the  system  is  probably 
correct  or  that  its  responses  are  credible  in  all  situations. 

When  building  a  human  behavior  model,  four  characteristics  are  key.  The  human  behavior  model  should 
be  usable,  robust,  flexible,  and  possess  the  ability  to  integrate  learning  and  self-explanatory  capabilities. 
A  usable  model  is  one  that  is  easily  learned  and  manipulated  by  the  model  user  for  model  set-up, 
modification,  and  building.  A  robust  model  responds  appropriately  to  unplanned  and  unexpected  events 
with  a  mechanism  that  allows  for  situation  assessment,  problem  solving,  and  planning,  and  provides  for 
graceful  degradation  of  model  capabilities.  A  flexible  model  is  one  that  will  allow  the  incorporation  of 
doctrine,  tactical,  and  knowledge  changes  in  a  manner  that  produces  behaviors  that  are  appropriate  within 
the  simulation  context  but  not  outwardly  predictable.  Finally,  a  model  that  possesses  a  self-explanatory 
capability  has  behaviors  that  are  easily  marked  within  the  software  when  they  are  invoked  and  the 
explanation  of  the  conditions  leading  to  the  invocation  of  the  behavior,  the  behavior  action,  and  the 
consequences  of  the  behavior  are  easily  traceable,  given  an  appropriate  set  of  software  tools. 
Experimentation,  in  terms  of  numbers  of  subjects,  variety  of  subjects,  and  repetitions  of  the  experiment, 
is  crucial  for  developing  these  four  desired  characteristics  for  a  human  behavior  model.  The  VI  offers  a 
means  for  achieving  the  breadth,  scope,  and  variability  required  in  order  to  construct  human  behavior 
models.  These  issues  and  concerns  pertain  to  the  development  of  team,  group,  and  cultural  behavioral 
models. 
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The  multi-national  research  experiment  that  serves  as  the  demonstration  vehicle  for  the  prototype  VI/HBR 
must  investigate  the  impact  of  culture  on  teamwork  in  both  national  and  multi-national  venues. 
The  experimentation  involves  conceptual  model  development  encompassing  the  impact  of  culture  on  joint 
military  operations,  extending  experimentation  methods  to  include  process  and  outcome  measures,  and 
identifying  products  for  national  use  to  improve  leader  and  team  performance  in  multi-cultural  environments. 
This  experimentation  will  be  executed  using  a  research  testbed  built  around  a  commercial  off  the  shelf 
(COTS)  computer  game,  Neverwinter  Nights™,  which  is  an  inexpensive,  standardized,  research  instrument 
able  to  explore  basic  research  questions  concerning  teamwork  skills,  situation  awareness,  decision  making, 
task  effectiveness,  adaptability,  and  the  impacts  of  personality  and  cultural  traits  on  these  issues.  A  single 
experimental  design  will  be  executed  in  all  the  participating  nations  to  build  a  baseline  of  responses  from 
homogenous  military  teams.  Then,  that  design  will  be  executed  with  heterogeneous  teams  comprised  of 
military  officers  from  various  nations  participating  in  the  baseline  experimentation. 

The  HBR  Virtual  Institute  experiment  team  collaborated  with  two  NATO  RTO  groups  to  further  its 
experimental  planning:  RTG  SAS-050  and  an  HFM  Exploratory  Team.  The  VI/HBR  experiment  team 
utilized  the  main  research  product  of  SAS-050,  which  was  a  conceptual  model  of  Command  and  Control 
(C2).  The  NATO  VI/HBR  experiment  team  analyzed  and  applied  this  conceptual  model  and  taxonomy  of  C2 
variables  as  a  starting  point  for  the  variables  within  the  cultural  modeling  testbed  to  measure  for  their  effects 
on  teamwork  and  cultural  influences  to  teamwork  and  leadership.  In  cooperation  with  an  HFM  Exploratory 
Team,  SAS-053  collaborated  on  the  experimental  approach  to  test  leader  and  team  adaptability  in  multi¬ 
national  coalitions  for  baseline  testing,  measurement,  and  analysis.  This  experimental  plan  and  approach 
allows  the  VI/HBR  prototype  and  the  testing  results  to  be  utilized  in  two  important  ways: 

1)  The  results  may  be  replicated  and  validated  through  additional  work  outside  the  current  multi¬ 
national  research  groups;  and 

2)  The  experimental  process  may  be  used  to  implement  additional  national  research  that  will  be  of 
comparable  use  across  nations  by  incorporating  the  experimental  and  testing  process  enabled  by 
the  virtual  institute  prototype. 

The  experiment  will  also  demonstrate  the  utility  and  efficiency  to  be  gained  by  using  a  virtual  institute  and 
its  infrastructure  to  conduct  experiments. 

3.1  HUMAN  BEHAVIOR  MODELING  BACKGROUND 

Human  behavior  modeling  is  the  development  of  computable  models  of  human  responses  to  situations  that 
are  realistic,  believable,  flexible,  responsive,  and  operate  at  an  interactive  rate.  The  quality  of  the  human 
behavior  model  directly  affects  the  fidelity  of  the  environment.  Human  behavior  modeling  is  not  a  simple 
or  straightforward  task  as  it  requires  the  acquisition  of  both  the  domain- specific  knowledge  and  the 
domain  independent  knowledge  that  humans  use  in  problem  solving.  As  the  basis  for  selecting  the 
parameters  for  human  behavior  modeling,  is  important  to  remember  that  the  behavior  models  must  reflect 
each  nation’s  characteristics.  Many  nations  have  undertaken  the  task  of  building  a  variety  of  models  for 
use  by  computer-generated  actors  that  reflect  their  own  national  behaviors.  The  development  of  these 
models,  their  variants,  challenges,  and  technological  challenges  to  be  overcome  can  be  addressed  in  the 
VI.  When  modeling  individual  human  behaviors,  there  are  a  large  number  of  parameters  and  factors  to  be 
considered  and  portrayed.  Individual  user  models  are  knowledge  representations  to  depict  users’ 
knowledge  and  interactions  with  a  computer  system.  The  main  purpose  of  user  modeling  is  to  determine 
what  the  user  intends  to  do  within  a  system’s  environment  for  the  purpose  of  assisting  the  user.  In  this 
context,  human  (user)  intent  may  be  defined  as  “mental  states”  that  drive  actions.  Intent  modeling  is  based 
on  the  belief  that  what  a  user  intends  to  do  in  an  environment  is  the  result  of  events  occurring  in  the 
environment  and  the  goals  he/she  is  trying  to  obtain  as  a  reaction  to  stimuli.  These  goals  can  be  explicit  or 
implicit,  physical  or  cognitive.  To  achieve  a  goal,  a  user  must  perform  certain  actions.  Goals  can  be 
composed  of  multiple  actions,  with  many  pre-  and  post-conditions.  Pre-conditions  include  directly 
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observable  events  in  the  environment  as  well  as  indirectly  observable  events  that  cause  a  user  to  pursue  a 
goal.  User  models  are  particularly  useful  in  domains  with  a  heterogeneous  group  of  users  and  where  the 
system  may  exhibit  flexibility  in  its  “response”  to  users. 

Human  behavior  modeling  also  benefits  from  researchers  in  the  fields  of  artificial  intelligence  (AI), 
human-computer  interaction  (HCI),  psychology,  education,  and  others  as  they  have  all  investigated  ways 
to  construct,  maintain,  and  exploit  user  models.  This  infusion  from  many  disparate  research  fields  has 
allowed  user  modeling  to  advance  by  exploiting  contributions  from  each  of  the  separate  contributing 
research  fields.  For  example,  the  user  modeling  community  has  been  able  to  reap  the  benefits  of  AI 
research  by  using  various  AI  knowledge  representations  such  as  logic-based  techniques,  abductive 
reasoning  techniques,  machine  learning  techniques,  Bayesian  methods,  and  neural  networks.  The  HCI 
research  field’s  impact  on  user  modeling  can  be  seen  in  the  use  of  user  models  to  customize  presentation 
of  information  to  provide  feedback  to  users  about  their  knowledge  in  a  domain  and  to  help  users  locate 
useful  information.  Another  HCI  impact  is  to  employ  lessons  learned  from  user  modeling  to  impact  the 
way  we  view  interactive  human-computer  environments.  This  approach  proposes  examining  interactive 
HCI  environments  along  three  orthogonal  dimensions: 

1)  Elements  -  the  goals,  plans,  resources,  and  actions  composing  the  atomic  entities  an  agent  (human 
or  otherwise)  is  concerned  with; 

2)  Processes  -  the  types  of  processing  (e.g.  reaction,  deciding,  learning)  that  takes  place;  and 

3)  Relationships  -  the  way  agents  interact  with  one  another. 

This  approach  makes  explicit  the  reasoning  about  the  purpose  of  adaptations  (why  adapt?),  treats  human 
and  computer  agents  the  same  in  the  environment,  takes  into  account  user  motivation,  emotions,  and 
moods,  and  presents  a  unified  model  of  collaborative,  cooperative,  and  adverse  behavior.  Additionally, 
user  models  have  taken  into  account  a  user’s  psychological  ability,  such  as  working  memory  or  cognitive 
load,  to  adapt  a  user  interface  and/or  the  information  presented  to  the  user. 

The  two  user  models  of  primary  interest  in  the  military  HBM  community,  behavioral  and  cognitive 
models,  are  both  performance  models  in  that  they  are  used  to  determine  a  user’s  future  actions. 
The  primary  difference  between  the  two  models  lies  in  the  level  to  which  the  user  is  modeled.  Both 
models  observe  the  user’s  execution  of  actions;  however,  cognitive  models  then  attempt  to  determine  the 
user’s  goals,  whereas  the  behavioral  model  directly  forecasts  user  activity.  Human  cognitive  models  have 
been  studied  by  researchers  in  the  field  of  psychology  for  many  years.  Cognitive  psychology  is  concerned 
with  understanding  tasks  in  which  a  stimulus  is  processed  in  some  way  before  a  response  is  chosen. 
Humans  form  cognitive  models  of  their  environment  to  make  sense  of  and  organize  the  information  they 
observe.  Cognitive  models  represent  aspects  of  users’  understanding,  knowledge,  intentions  and 
processing,  and  tend  to  have  a  computational  flavor.  Stokes  emphasizes  that  to  be  adaptive,  the  system 
requires  a  model  of  the  cognitive  state  of  the  human  operator  that  will  infer  both  the  present  level  of 
operator  performance  and  the  current  state  of  mental  workload  and  resource  allocation.  A  behavioral 
model  is  the  other  performance  type  of  user  model  and  attempts  to  address  some  of  the  difficulties 
encountered  when  using  cognitive  user  model.  In  a  behavioral  model,  the  behavior  of  a  system  is 
manifested  in  input-output  relationships;  the  user’s  behavior  can  be  defined  as  a  succession  of  states. 
Put  another  way,  a  behavioral  model  represents  the  human  user  as  a  collection  of  sequences  of  actions  that 
the  user  performs.  This  model  observes  and  predicts  the  motions  of  the  user.  The  system  does  not  attempt 
to  determine  the  user’s  goal,  as  done  with  a  cognitive  model,  but  directly  predicts  future  user  actions  based 
on  the  status  of  the  environment  and  past  user  actions. 

The  first  step  in  the  development  of  a  user  model  involves  examining  the  tasks  users  perform  and  the 
knowledge  they  use  to  perform  these  tasks  and  then  representing  the  knowledge.  One  approach,  cognitive 
task  analysis  (CTA),  centers  on  informing  the  design  process  through  the  application  of  cognitive  theories. 
A  task  is  defined  as  what  the  person  or  other  intelligent  agent  has  to  do  (or  believes  is  necessary) 
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to  accomplish  a  goal  by  use  of  some  device.  Hierarchical  task  analysis  is  concerned  with  establishing  an 
accurate  description  of  the  steps  that  are  required  to  complete  a  task,  the  focus  of  CTA  is  on  techniques 
that  capture  some  representation  of  the  knowledge  that  people  have,  or  that  they  need  to  have,  to  complete 
the  task.  The  underlying  assumption  of  much  of  cognitive  psychology  is  that  a  human  perceives  the  world 
and  produces  some  representation  of  it  in  his  or  her  mind,  called  the  “problem  space”.  This  representation 
is  what  we  would  usually  call  “knowledge”.  This  knowledge  may  be  described  in  terms  of  the  concepts 
that  we  possess,  the  relationships  between  those  concepts  and  our  capacity  to  make  use  of  those  concepts. 
The  human  then  manipulates  that  representation  and  produces  some  output,  or  behavior,  that  can  be 
observed.  The  two  principle  levels  of  cognitive  activity  that  must  be  undertaken  within  a  user  centered 
design  framework  are  the  task-action  representations  and  mappings  and  the  goal-task  representations  and 
mappings.  Of  the  various  cognitive  models,  the  most  important  historically  is  the  model  human  processor 
(MHP),  which  assumes  that  a  psychological  model  consists  of  three  interacting  systems:  the  perceptual, 
motor,  and  cognitive  systems.  Each  of  these  systems  has  its  own  memory  (maintains  an  internal 
representation  or  knowledge)  and  processor.  This  model  led  to  the  GOMS  (Goals,  Operations,  Methods, 
and  Selection  rules)  method  of  CTA.  Johnson’s  theory  of  Task  Knowledge  Structures  (TKS)  assumes  that 
as  people  learn  and  perform  tasks,  they  develop  knowledge  structures.  He  utilizes  method  known  as 
Knowledge  Analysis  of  Tasks  (KAT)  to  identify  the  elements  of  knowledge  represented  in  a  task 
knowledge  structure.  Other  cognitive  task  analysis  techniques  that  focus  on  different  aspects  of  the  general 
information  processing  model  include  the  Task  Action  Grammar  (TAG),  which  is  concerned  with  an 
evaluation  of  the  leamability  of  systems;  and  Moran’s  External  Task  Internal  Task  (ETIT)  and  Payne’s 
Yoked  State  Space  (YSS),  which  are  concerned  with  the  mapping  of  tasks  from  the  external  task  space  to 
the  internal  task  space.  The  papers  by  Rosenbloom,  and  Tambe  present  a  brief  overview  of  capabilities  for 
the  TacAir-SOAR  system.  Based  on  the  SOAR  architecture,  the  system  capabilities  include  a  knowledge 
base  for  the  air  combat  domain,  improvements  in  its  behavior,  and  infrastructure  enhancements  from  the 
architectural  basis.  Improvements  in  the  TacAir-SOAR  domain  capabilities  focused  on  improving  the 
robustness  and  range  of  scenarios  it  can  operate  within  and  increasing  the  number  of  TacAir-SOAR 
entities  within  the  DVE.  Robustness  and  range  of  capabilities  were  improved  via  the  addition  of  rules  to 
permit  participation  in  a  wider  number  of  missions  and  the  capability  to  interact  with  three  other  aircraft. 
The  authors  note  that  the  most  significant  advance  in  the  system’s  capabilities  was  the  addition  of  the 
ability  to  maintain  episodic  memories  of  engagements  and  to  use  these  memories  to  explain  its  actions. 

These  representations  of  cognitive  activity  rely  upon  reasoning  systems  to  supply  the  decision-making 
system  required  to  portray  human  behavior.  There  are  a  number  of  systems  that  can  be  used  as  reasoning 
systems.  Prepositional  logic  assists  this  reasoning  by  providing  a  mechanism  that  enables  us  first  to 
evaluate  simple  statements  and,  subsequently,  complex  statements  formed  through  the  use  of  prepositional 
connectives,  like  and  and  or.  This  mechanism  determines  the  truth  of  a  statement  from  the  truth-values 
assigned  to  the  original  simple  statements.  Predicate  logic  is  an  extension  of  prepositional  logic  that  allows 
more  flexibility  and  power  in  the  representation  of  knowledge.  Predicate  logic  permits  the  expression  of 
relationships  between  objects  or  abstract  quantities.  Predicates  are  these  relationships,  and  their  arguments 
are  the  objects  that  these  relationships  tie  together.  Predicate  logic  also  supports  the  use  of  variables, 
which  bind  a  predicate’s  arguments  to  successively  different  values.  Rules  represent  conditional 
knowledge  that  is  similar  to  the  way  human  express  it.  Rules  express  knowledge  as  a  two-part 
relationship.  The  first  part  is  a  conditional  test,  called  the  IF,  the  premise ,  or  the  antecedent.  If  the  test  is 
satisfied  through  a  true  match  with  known  facts,  then  the  second  part,  called  the  THEN ,  the  actions ,  or  the 
consequent ,  is  executed.  Rules  look  like  conditional  statements  in  conventional  languages,  but  unlike  the 
latter,  rules  are  applied  in  a  totally  different  manner,  making  rule-based  systems  declarative.  Rules  are  by 
far  the  most  popular  knowledge  representation  technique  within  existing  systems.  This  use  can  be 
attributed  to  the  excellent  ability  of  rules  to  represent  heuristic  knowledge  and  ease  of  implementation  and 
under standability.  A  semantic  or  associative  network  is  a  labeled,  directed  graph.  The  nodes  in  the 
network  are  used  to  represent  various  concepts  or  objects,  and  the  arcs  or  links  connecting  the  nodes 
represent  the  various  relationships  or  associations  that  hold  between  the  concepts.  The  important  feature  of 
the  associative  network  is  the  associative  links  that  connect  the  various  nodes  within  the  network  and 
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define  associative  graphs  apart  from  simple  directed  graphs.  These  links  define  relationships  that  provide 
the  structure  for  understanding  the  meaning  represented  to  define  a  particular  concept  and  provide  a 
cohesive  structure  for  interpreting  the  structure  and  inferring  new  facts.  By  representing  knowledge 
explicitly  within  an  associative  network,  a  system  obtains  a  higher  level  of  understanding  for  the  actions, 
causes,  and  events  that  occur  within  the  domain.  Associative  networks,  rules,  and  logic  do  not  provide  the 
ability  to  group  facts  into  associated  clusters  or  to  associate  relevant  procedural  knowledge  with  some  fact 
or  group  of  facts.  Frames  attempt  to  account  for  the  human  ability  to  deal  with  new  situations  by  using 
existing  knowledge  of  previous  events,  concepts,  and  situations.  A  frame  provides  the  structure  or 
framework  for  representing  the  knowledge  acquired  through  previous  experience.  Using  a  collection  of 
knowledge  consisting  of  a  frame  name  and  a  set  of  attribute-value  pairs,  a  frame  represents  a  stereotypical 
situation  from  the  world.  The  attributes  of  the  attribute-value  pair  are  often  called  slots  while  the  values 
are  called  fillers.  The  fillers  can  additionally  be  subdivided  into  facets,  each  having  their  own  associated 
values.  Unlike  frames,  the  object  approach  creates  a  tight  bond  between  the  code  and  data  rather  than 
separating  them  into  two  complex,  separate  structures.  Small  chunks  of  code  and  data  are  tightly  coupled 
together  with  each  of  these  chunks  loosely  tied  to  each  other. 

While  there  exist  many  mechanisms  to  manipulate  knowledge  representation  techniques,  the  only  ones  in 
current  operational  use  within  the  simulation  community  are  techniques  dealing  with  logic  and  rules. 
Prepositional  logic  and  predicate  logic  both  serve  as  an  implementation  of  automated  reasoning.  Logic 
defines  sound  rules  of  inference  that  allow  the  logical  inference  of  new  truths  from  existing  truths.  The  most 
significant  of  these  rules  of  inference  are  modus  ponens,  modus  tolens,  and  resolution.  Modus  ponens  forms 
the  basis  for  deductive  reasoning,  which  is  reasoning  from  known  facts  to  unknowns.  Deduction  always 
guarantees  a  true  conclusion  if  the  initial  facts  or  premises  are  true.  Deduction  is  the  most  widely  accepted  as 
well  as  recognized  reasoning  method.  Another  method  of  logic  reasoning  is  called  abduction.  Abduction 
attempts  to  derive  a  rule’s  premises  from  observing  its  conclusions.  A  final  example  of  logic  reasoning  is 
called  induction,  which  forms  the  basis  for  learning.  Induction  is  performed  by  observing  facts  and 
generalizing  from  them.  Rules  are  applied  in  two  ways:  data  driven  and  goal  driven.  If  the  number  of  inputs 
is  limited,  many  inputs  are  acquired  automatically,  and/or  the  number  of  possible  conclusions  is  large, 
then  rules  are  applied  in  a  data-driven  fashion.  Alternately,  if  the  number  of  possible  final  conclusions 
(called  goals)  is  limited,  and/or  input  values  are  not  acquired  automatically,  then  reasoning  backwards  from  a 
goal  is  more  efficient.  In  goal-driven  reasoning,  or  backward  chaining,  the  rules  are  applied  only  to  derive 
values  for  goals  or  for  intermediate  facts  used  later  to  set  values  to  these  goals. 


3.2  HUMAN  BEHAVIOR  MODELING  PARAMETERS 

3.2.1  Parameters  for  Individual  Modeling 

The  area  of  individual  human  behavior  modeling  has  witnessed  a  wide  variety  of  approaches.  In  general, 
researchers  advocate  that  a  model  of  human  behavior  should  account  for  attention,  intent,  situation 
assessment,  decision-making,  skills,  training,  fatigue,  environmental  stressors,  education,  experience, 
and  motivations.  To  date,  very  few  systems  have  attempted  to  model  more  than  a  few  of  these  factors  and 
most  models  are  generally  assembled  using  an  ad  hoc  approach  instead  of  employing  a  strong 
methodology.  For  example,  human  behavior  has  been  modeled  by  separating  it  into  perception,  cognition, 
and  action  as  well  as  a  hierarchy  of  task  flows  and  cognition.  Early  efforts  at  human  behavior  modeling 
used  finite  state  machines;  recently,  these  efforts  have  been  coupled  with  agents  to  try  to  improve  the 
fidelity  of  the  modeled  behaviors.  Concurrent  control  and  cognition  in  conjunction  with  arbitration  as  well 
as  case-based  reasoning  coupled  with  machine  learning  have  also  been  investigated.  Intelligent  agents 
have  also  been  used  to  model  behavior  by  having  one  agent  responsible  for  each  type  of  behavior  and  then 
arbitrating  or  scaling  their  individual  outputs.  Early  implementations  generally  relied  upon  a  blackboard 
approach  to  model  cognition.  Recently,  neural  networks  and  fuzzy  logic  have  both  been  evaluated  for  their 
ability  to  capture  and  portray  a  wide  range  of  diverse  human  behaviors.  Recent  work  in  human  behavior 
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modeling  for  CGAs  has  addressed  situation  awareness,  teamwork,  skill  level,  training,  learning, 
and  psychological  factors  to  a  limited  degree.  Models  of  the  effects  of  fatigue  have  been  developed, 
but  most  of  these  efforts  have  been  ad  hoc  and  have  not  produced  a  methodology  or  structure  for  the 
model.  Planning  has  been  modeled  as  a  set  of  competing  processes  that  gradually  progress  toward  a  final 
solution  as  additional  information  becomes  available.  The  cognitive  models  that  have  been  reported  use  a 
variety  of  techniques  such  as  motivational  control,  beliefs-desires-intentions,  and  traits  and  states. 
The  level  of  training  is  modeled  by  determining  the  skills  relevant  to  a  task  and  assigning  the  level  of  skill 
given  the  level  of  training  to  be  portrayed. 

In  general,  an  examination  of  the  human  behavior  modeling  research  reveals  that  the  work  is  fragmented 
and  that  research  results  are  difficult  to  integrate  and  validate.  Currently,  there  is  no  comprehensive  theory 
or  modeling  approach  for  integrating  the  various  efforts  or  to  serve  as  a  common  platform.  The  results  are 
generally  not  portable;  that  is,  the  software  developed  for  a  given  architecture  cannot  be  readily  migrated 
to  another  software  platform.  The  results  are  also  not  generally  in  use;  research  results  have  been  confined 
to  the  laboratory  and  have  not  moved  into  wide  use.  Finally,  the  models  are  difficult,  costly,  and  time 
consuming  to  assemble.  To  improve  the  fidelity  of  human  behavior  models,  they  must  incorporate 
multiple  skill  levels,  representations  of  doctrine,  and  reflect  societal,  training,  and  educational  differences. 
Additional  behavior  moderators  such  as  morale,  fatigue,  stress,  fear,  information  operations,  and  intent 
must  be  modeled  and  included  within  the  model. 

When  attempting  to  model  individuals,  a  wide  variety  of  parameters  must  be  considered  in  order  to  portray 
a  human  in  a  specific  situation  or  set  of  circumstances.  The  number  of  parameters  needed  to  model  human 
behavior,  uncertainties  concerning  the  number  of  parameters  that  are  required,  and  uncertainties  about  the 
interplay  between  parameters  have  resulted  in  a  large  number  of  approaches  and  parameter  sets  for 
modeling  human  behavior.  An  individual  behavior  model  must  account  for  a  number  of  factors  that  affect 
the  human’s  behavior,  including  skill  level  representations,  stress  factor  representations,  different  levels/ 
abilities  of  performance,  and  situational  psychological  aspects.  The  LaVine,  Laughery,  Young,  and  Kehlet 
taxonomy  consists  of  five  skills,  which  are  attention,  perception,  psychomotor,  physical,  and  cognitive 
skills.  In  their  model,  attention  is  the  ability  to  attend  to  a  stimulus  for  extended  periods.  Perception  is  the 
ability  to  detect  and  categorize  stimuli.  The  psychomotor  skill  models  the  ability  to  maintain  select 
characteristics  of  an  environment  within  a  desired  range  of  conditions.  Physical  skill  models  the  ability  to 
perform  muscular  work.  Cognitive  skill  is  the  ability  to  apply  concepts  and  rules  to  the  environment  to 
generate  plans  and  actions.  The  ability  to  perform  each  skill  is  rated  on  a  seven-point  scale  by  a  subject 
matter  expert  using  the  questionnaire.  Weaver  and  Mullen  address  the  subject  of  modeling  military 
decision-making  using  a  cognitive-type  model.  Their  approach  to  decision-making  modeling  is  composed 
of  four  parts:  the  intrinsic  characteristics  of  the  decision,  the  perceived  environment,  a  neural  network 
(for  decision-making),  and  three  output  variables  that  scale  the  decision.  The  two  external  inputs  into  the 
decision  are  the  intrinsic  classification  characteristics  of  the  decision  and  the  environment  for  the  decision. 
To  characterize  the  decision  these  data  are  then  combined  using  a  model  of  human  decision-making 
implemented  in  a  neural  network.  Their  system  has  three  output  variables;  the  first  of  the  variables  is  the 
timeliness  of  the  decision,  the  second  is  the  quality  of  the  decision,  and  the  third  output  is  the  quantitative 
aspect  of  the  decision. 

There  are  many  differences  that  can  be  noted  in  the  manner  in  which  individuals  and  teams  perform 
decision-making,  as  summarized  in  Table  3-1.  For  example,  individual  decision-making  may  have  a  much 
narrower  focus  and  may  not  consider  all  relevant  data  when  compared  to  team  decision-making. 
Additionally,  team  decision-making  is  usually  driven  by  a  desire  to  build  a  degree  of  consensus;  whereas, 
individual  decision-makers  do  not  have  that  obstacle.  In  addition,  apparently  it  is  more  difficult  for  a  team 
than  an  individual  to  assemble  a  consistent  mental  model  of  the  battlespace  but  teams  are  more  likely  to  have 
a  broader  appreciation  for  the  complexities  of  the  battlespace  than  an  individual’s  inherent  limitations  allow 
them  to  attain.  Experimentation  has  determined  that  individuals  are  limited  to  being  able  to  accommodate 
(attend  to)  seven  (plus  or  minus  two)  items  in  mind  at  any  instant.  No  equivalent  formulation  that  describes  a 
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team’s  attention  has  been  developed;  but  it  would  seem  to  be  greater  due,  no  doubt,  to  the  ability  of  the 
people  in  the  team  to  overlap  and  reinforce  items  to  which  their  individual  concerns  are  addressed 
(or  attended).  On  the  other  hand,  individuals  can  readily  attend  to  one  item  and  exclude  all  other  matters 
from  attention;  individuals  have  a  well-documented  ability  to  focus  their  attention  on  one  item  to  the 
exclusion  of  all  others.  Conversely,  it  is  apparently  very  difficult  for  an  entire  team  to  focus  its  attention  on 
one  item  (and  in  many  circumstances  it  is  counterproductive  to  do  so),  providing  both  the  promise  that 
important  activities  are  not  overlooked  but  also  threatening  to  dilute  the  effectiveness  of  the  actions  and 
decisions  of  the  team.  The  difference  in  ability  to  attend  to  one  matter  versus  several  corresponds  to  the 
difference  in  the  number  of  active  decision  processes  in  an  individual  and  a  team  at  any  time.  Generally, 
an  individual  can  pursue  one  active  decision  process,  whereas  a  team  can  have  several.  Interestingly, 
the  literature  suggests  that  both  individuals  and  teams  are  subject  to  the  problem  of  process  interruption;  that 
is,  when  a  process  is  underway  (whether  it  is  a  decision  or  information  flow)  and  it  is  interrupted,  it  is  a 
challenge  for  the  team  or  individual  to  recover  from  the  interruption.  Though  it  appears  that  teams  may  be 
able  to  recover  more  often  than  individuals,  teams  are  also  subject  to  more  catastrophic  failures  when 
interrupted  during  the  execution  of  a  task  than  are  individuals  due  to  the  need  for  communication  and  a 
shared  understanding  of  the  state  of  task  accomplishment. 


Table  3-1 :  Individual  versus  Team  Characteristics  for  Information  Management  and  Decision-Making 


Characteristic 

Individual 

Team 

Scope  of  detail 

Limited,  difficult  to  consider/ 
attend  to  many  details 
simultaneously 

Broad,  many  details  can  be 
considered  simultaneously 

Breadth  of  attention 

Limited,  7  +/-  2 

Unknown,  but  seems  to  scale 
with  size  of  team 

Focus  of  attention 

Can  be  highly  focused 

Counter-productive  to  focus 
on  one  item 

Rapidity  of  decision¬ 
making 

Personality  dependant 

Team  competence  and 
composition  dependant 

Number  of  active  decision 
processes 

One  or  few 

Multiple,  no  generally 
accepted  model 

Fatigue 

One  level 

No  certain  model 

Decision-making  model 

OODA 

None 

Decision-process 

interruption 

Problem 

Problem 

In  general,  an  individual  has  one  skill  level  and  has  one  fatigue  level  at  a  given  time  and  they  change 
relatively  slowly.  As  a  result,  it  is  possible  to  characterize  a  person’s  response  to  events  and  potentially 
determine  how  to  improve  individual  performance  when  fatigued,  under  stress,  or  when  performance  does 
not  meet  expectations.  On  the  other  hand,  a  team  has  multiple  skill  levels  in  effect  concurrently  as  well  as 
multiple  fatigue  factors  and  it  is  not  clear  how  to  combine  them  or  ameliorate  their  effects  in  order  to 
improve  team  performance.  Finally,  an  individual  remains  relatively  “constant”  in  their  performance  over 
the  duration  of  an  experiment,  a  team  does  not  appear  to  remain  “constant”  since  the  team  can  vary 
(in  composition  and  therefore  “skill”  and  “performance”,  as  well  as  in  group  dynamic  and  teamwork)  over 
the  duration  of  a  simulation  experiment. 
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3.2.2  Parameters  for  Team  Modeling  and  Group  Modeling 

As  the  basis  for  selecting  parameters  for  team  and  group  modeling,  is  important  that  the  representation  of 
individual  nations  in  a  simulation  environment  reflect  each  nation’s  characteristics.  Many  nations  have 
undertaken  the  task  of  building  a  variety  of  models  for  use  by  computer  generated  actors  that  reflect  their 
own  national  behaviors.  The  development  of  these  models,  their  variants,  challenges,  and  technological 
challenges  to  be  overcome  can  be  addressed  in  the  VI. 

In  the  ITEMS  (Interactive  Tactical  Environment  Management  System)  system  described  by  Siksik,  models 
systems  and  intelligence  for  actors  within  an  environment  and  was  designed  to  simulate  Army  company- 
level  activities  within  a  battle  space.  The  ITEMS  approach  to  achieving  realistic  behavior  is  the  use  of  expert 
system  technology  to  provide  the  foundation  for  a  CGA’s  intelligence.  In  ITEMS,  actors  are  assembled  from 
libraries  that  specify  sensors,  weapons,  communications,  dynamics,  and  knowledge.  To  perform  decision¬ 
making,  ITEMS  uses  rule-based  expert  systems  and  frames  and  divides  the  decision  process  into  four  types: 

1)  Mission; 

2)  Opponent  selection; 

3)  Player  action;  and 

4)  Command  and  control. 

The  player  action  component  contains  the  knowledge  needed  to  allow  the  CGA  to  perform  its  mission  and 
to  react  to  opponents  and  external  stimuli,  and  can  be  specialized  into  air  combat,  company,  and  battalion 
knowledge  bases.  Within  ITEMS,  knowledge  is  encapsulated  in  frames,  with  each  frame  responsible  for 
representing  facts  about  a  specific  type  of  knowledge.  To  reduce  the  knowledge  representation  problem, 
inheritance  is  used  to  permit  specialization  of  knowledge  from  a  frame  that  represents  general  knowledge. 
In  the  ITEMS  expert  system  approach,  there  are  three  separate  knowledge  representations: 

1)  A  database; 

2)  A  rules  database;  and 

3)  A  common  database. 

The  object-oriented  database  holds  all  of  the  information  required  to  represent  a  scenario  and  it  holds  the 
rules  database.  The  scenario  definition  portion  of  the  database  holds  modular,  frame-based  libraries 
organized  in  a  hierarchical  structure  that  supports  re-use  of  libraries  at  any  level  of  the  hierarchy. 
Each  library  is  responsible  for  representing  a  single  type  of  knowledge/information  needed  to  define  a 
scenario.  The  rules  portion  of  the  database  is  also  structured  as  re-usable  frame-based  libraries.  Each  of  the 
rules  libraries  is  responsible  for  representing  the  complete  set  of  knowledge  needed  to  support  decision¬ 
making  for  motion,  navigation,  communication,  tactics,  and  mission  execution.  Each  library  is,  in  turn, 
composed  of  frame-based  records  that  contain  all  of  the  information  needed  to  make  a  specific  type  of 
decision.  Each  record  is,  in  turn,  composed  of  rules  and  state  descriptions.  The  database  holds  transient 
knowledge  concerning  the  state  of  the  simulation  environment.  The  ITEMS  project  devoted  considerable 
effort  to  the  modelling  of  command  and  control  structure  in  the  belief  that  correct  modelling  of  command 
and  control  would  lead  to  realistic  behaviors  on  the  part  of  the  CGAs  that  it  controls.  The  ITEMS 
approach  seems  more  oriented  toward  instantiation  of  formations,  such  as  companies  and  battalions,  than 
of  individual  actors.  This  system  provides  the  basis  for  the  parameter  set  developed  within  SAS-053. 
ITEMS  point  towards  a  number  of  team  performance  factors. 

When  examining  and  assessing  team  performance,  a  number  of  factors  related  to  both  the  individuals  on 
the  team  as  well  as  the  team  entire  must  be  simulated  in  order  to  properly  model  the  team;  however,  it  is 
imperative  that  the  team’s  effectiveness  be  expressed  in  relationship  to  each  of  its  goals  and  not  in  the 
aggregate.  The  individual  parameters  were  discussed  in  the  preceding  sub-section,  at  this  point  we  will 
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limit  the  discussion  to  team  specific  factors.  The  team’s  situation  awareness  and  its  perceptions  must  be 
modeled  along  with  the  team’s  ability  to  work  together.  Because  the  team’s  ability  to  work  together  is  an 
important  modeling  parameter,  the  effect  of  cultural  diversity  upon  team  performance  is  important  and 
should  be  explicitly  modeled,  which  indicates  that  a  framework  for  experimentation  and  expressing  the 
impact  of  cultural  diversity  should  be  developed  (possibly  in  conjunction  with  the  HFM  Panel).  Two  more 
factors  to  be  considered  and  modeled  is  the  affect  of  individual  perception  and  individual  situation 
awareness  upon  team  perception  and  team  situation  awareness  and  how  these  factors  affect  team 
performance.  When  conducting  team  performance  experiments  further  factors  need  to  be  measured  and 
modeled  in  order  to  assess  the  team,  and  each  of  these  factors  must  be  varied  for  each  individual  in  the 
team  in  order  to  determine  all  of  the  variations  in  team  behavior.  These  factors  include  the  average  data 
volume  to  an  individual  at  each  point  in  the  team’s  activity  operational  scenario,  the  peak  data  volume  to 
an  individual  during  the  scenario,  the  data  accuracy  to  an  individual,  the  average  taskload  to  an  individual 
during  “normal”  conditions  in  the  team’s  operation,  and  the  peak  taskload  to  an  individual  during 
“normal”  conditions.  Additional  considerations  include  the  average  taskload  to  an  individual  during 
“stress”  or  “attack”  conditions,  the  peak  taskload  to  an  individual  during  “stress”  or  “attack”  conditions  on 
the  team,  the  average  number  of  personal  contacts/communications  outside  of  team  during  “normal” 
conditions,  and  the  average  number  of  personal  contacts/communications  outside  of  the  team  during 
“stress”  or  “attack”  conditions.  Also,  for  each  person  in  the  team  the  degree  of  responsibility/autonomy  for 
the  individual,  the  level  of  situation  awareness  required  by  the  individual  in  order  to  properly  perform  their 
team  duties,  and  the  degree  of  autonomy  that  each  individual  possesses  during  different  scenarios  and 
during  different  phases  of  the  scenario  must  be  assessed  and  modeled  in  order  to  simulate  the  team’s 
operation.  Team  cohesion  also  affects  team  performance  and  must  be  assessed  in  order  to  build  a  team 
model,  and  if  the  model  is  to  have  high  utility,  and  to  accurately  model  cohesion  and  team  performance, 
the  fidelity  and  parameters  for  individuals  and  sub-groups  must  be  varied  in  order  to  assess  the  entire 
spectrum  of  team  behaviors.  All  of  these  factors  come  together  in  a  command  post  setting  and  affects  the 
effectiveness  of  the  command  post’s  operation.  Team  cohesion  also  affects  team  performance  and  must  be 
assessed  in  order  to  build  a  team  model,  and  if  the  model  is  to  have  high  utility  the  fidelity  and  parameters 
for  individuals  and  sub-groups  must  be  varied  in  order  to  assess  the  entire  spectrum  of  team  behaviors. 

Command  posts  are  one  of  the  most  important  areas  for  team  modeling  because  of  their  centrality  to  mission 
accomplishment  and,  of  course,  command  and  control.  Within  the  design  of  command  post  experiments 
there  are  four  factors  that  affect  command  post  team  performance: 

1)  Sensors; 

2)  Intelligence  reports; 

3)  Network/communications  devices;  and 

4)  Personnel  who  make  decisions,  and  take  actions. 

All  four  factors  are  nearly  independent.  There  appears  to  be  an  inter-dependence  between  intelligence 
reporting  and  personnel  decision-making  in  command  post  team  performance.  The  research  literature  and 
analysis  indicates  that  there  are  two  broad  areas  of  command  post  team  evaluation  that  must  be  examined 
experimentally.  The  first  experimental  area  is  the  investigation  of  the  four  factors  separately.  The  second 
experimental  area  is  examination  of  the  interconnection  between  intelligence  and  personnel  decisions  and 
actions.  Fortunately,  the  two  areas  are  independent  enough  so  that  they  can  be  examined  simultaneously; 
i.e.  one  experiment  can  test  hypotheses  for  both  areas  simultaneously  without  corrupting  the  experiment 
results  for  either  area.  When  attempting  to  gain  insight  into  the  first  area,  experiments  must  be  performed 
to  determine  how  to  improve  each  of  the  four  factors  that  affect  command  post  team  performance.  When 
attempting  to  gain  insight  into  the  second  area,  experiments  must  be  performed  to  determine  how  to 
improve  the  processes  for  gathering,  handling,  examining,  and  managing  intelligence  that  affect  command 
post  team  performance.  In  sum,  command  post  team  simulation  experiments  should  do  the  following: 
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1)  Concentrate  on  these  two  cases; 

2)  Assess  the  flow,  quality,  and  quantity  of  information; 

3)  Attempt  to  match  real-world  command  post  team  expertise,  numbers,  and  taskload,  especially  in 
the  areas  of  command  post  performance  that  are  central  to  the  experiment;  and 

4)  Attempt  to  match  the  equipment  available  in  the  command  post  being  simulated. 

For  both  cases,  quality  of  intelligence  reporting  may  be  a  factor  and  should  be  explicitly  considered  in  the 
experimental  design. 

The  research  literature  and  analysis  indicated  that  within  the  area  of  command  post  team  operations 
dealing  with  intelligence  that  there  are  two  distinct  cases  of  intelligence  input  to  consider.  The  first 
intelligence  case  revolves  around  determining  how  well  the  command  and  control  structure  and  process 
are  able  to  correlate  activities  in  the  environment  that  are  precursors  to  events.  A  mix  of  intelligence 
inputs  are  needed  because  the  experiment  is  actually  trying  to  determine  how  well  a  significant  event  can 
be  separated  from  background  information  (aka  noise).  False/inaccurate  intelligence  would  be  a  normal 
part  of  that  background,  so  the  ability  to  detect  and  reject  false  positives  as  well  as  insure  that  detection  of 
true  precursors  is  determined.  The  measures  of  command  and  control  (C2)  performance  that  must  be 
applied  in  this  case  are  those  that  assess  the  command  and  control  (C2)  structure’s  ability  to  determine  the 
accuracy/validity  of  the  information  that  it  has  and  how  well  a  given  C2  structure  and  processes  supports 
the  information  determinations  that  must  be  made.  The  second  intelligence  case  revolves  around 
determining  how  well  the  command  and  control  structure  and  process  is  able  to  move  information 
between  the  command  post  components  that  need  the  information  at  any  point  in  time.  In  this  experiment 
case,  the  goal  is  measuring  the  latency  between  the  time  when  information  becomes  available  until  all 
portions  of  the  command  post  structure  that  need  the  information  have  the  information. 

When  considering  experimental  area  one,  the  metrics  used  vary  depending  upon  the  intelligence  case 
being  considered.  For  intelligence  case  one  in  experimental  area  one,  potential  metrics  include  but  are  not 
limited  to  the  following: 

1)  Elapsed  time; 

2)  Number  of  participants  in  the  process; 

3)  Number  of  extraneous  participants  in  the  process; 

4)  Number  of  precursor  events  that  are  addressed  by  the  C2  structure; 

5)  Number  of  alternative  explanations  for  a  precursor  event  that  are  considered; 

6)  Criteria  applied  to  determine  the  significance  of  a  precursor  event; 

7)  The  number  of  times  that  the  explanation  assigned  to  a  precursor  event  by  a  given  C2  structure  is 
accurate; 

8)  The  “difficulty”  faced  when  determining  what  a  precursor  event  signifies; 

9)  The  number  of  times  a  sensor  is  re-tasked  correctly; 

10)  The  number  of  times  a  sensor  is  re-tasked  incorrectly  (unnecessarily); 

11)  The  “quality”  of  the  decision/interpretation;  and 

12)  The  overall  “quality”  of  decision  process. 

For  intelligence  case  two  in  experimental  area  one,  potential  metrics  include  the  following: 

1)  The  speed  of  the  movement  of  information  through  the  command  post  and  C2  structure; 
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2)  The  number  of  people  who  should  have  the  information  but  did  not  get  it  (directly); 

3)  The  number  of  people  who  did  not  need  the  information  but  received  it; 

4)  The  information  volume; 

5)  The  “quality”  of  communication; 

6)  The  number  of  times  that  information  has  to  be  retransmitted;  and 

7)  The  latency  between  when  a  re-tasking  directive  is  sent  and  when  the  re-tasking  is  accomplished. 

When  considering  experimental  area  two,  the  metrics  to  be  used  do  not  vary  depending  upon  the 
intelligence  case  being  considered.  For  this  experiment  area,  potential  metrics  include  but  are  not  limited 
to  the  following: 

1)  The  elapsed  time  to  make  a  decision; 

2)  The  “quality”  of  a  decision; 

3)  The  number  of  alternative  responses  considered; 

4)  The  “quality”  of  processes  employed; 

5)  The  number  of  people  involved  in  the  decision  process; 

6)  The  number  of  extraneous  participants  in  the  process; 

7)  The  criteria  applied  to  determine  a  response; 

8)  The  “difficulty”  of  determining  the  correct  response; 

9)  The  number  of  pieces  of  information  considered  when  formulating  a  response; 

10)  The  number  of  pieces  of  extraneous  information  considered  when  formulating  a  response; 

11)  The  overall  assessment  of  the  command  post’s  “comprehension”  of  the  battlespace;  and 

12)  The  ability  of  the  command  post  members  to  form  a  common,  shared  mental  model  of  the 
battlespace. 
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3.3  EXAMPLE  HUMAN  BEHAVIOR  MODELING  APPLICATIONS 


Modeling  Effects  Based 
Operations 

Liz  Bowman 
May  1 1 , 2006 
SAS-053  Meeting 


RTO-TR-SAS-053 


3-13 


STATE  OF  THE  ART  SUMMARY 


ORGANIZATION 


Agenda 

•  C3TRACE  architecture 

•  Introduction  to  MNE  4 

•  MNE  4  Model-Test-Model  Development 
-  Model:  The  expected  EBAO  model 
-Test:  Data  Collection  at  MNE  4 
-Model:  Model  Refinement 


2 


C3TRACE 

Components 


Communication  Flows  and  Content  Analysis 
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1  Perception 

2  Understanding 
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•  Time  for  task  completion 
•Workload  (V,C,A,P) 

•  Assignments  of  tasks  to 
individuals/groups 
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C3TRACE  Outputs 

•  Utilization  rates  by  persons/groups 

•  Periods  of  high  workload  and  associated  consequences: 
dropped  tasks,  delayed  communications,  missed 
communications 

•  Average  information  quality,  based  on  each 
communication’s: 

-  Time  since  update 

-  Volatility  of  information  (enemy  who/enemy  where) 

-  Decay  rate  of  information 

-  Original  Information  quality  (complete,  timely,  accurate) 

•  Probability  of  decision  quality  based  on  information 
quality 
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MNE  4  Review 


•  Coalition  HQ  (operational  level)  conducting 
support  operation  [planning,  execution,  assessment] 

•  Distributed  context:  participants  used  collaborative 
technology  to  execute  tasks 

-  Text  chat 

-  Audio 

-  Limited  Face  to  Face  meetings 

•  Concept  of  Operations  document  for  MNE  4  defined 
process  and  organization  required  for  experiment 
execution 


a  peace 


•  Technology  used  included  full-HQ  use  (IWS)  and 
functional  use  by  groups  (EBA  used  EBAT,  EBP  used 
EBTOPFAS) 


•  MNE  4  was  an  8  nation  coalition 
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MNE  4  Organization 


MNE  4  ConOps  Review 

The  ConOps  was  the  basis  for  the  Expected  MNE  4  Model 


Knowledge  Base  Development  (KBD) 


Effects  Based  Planning  (EBP) 


4  functions 

1 7  tasks 

41  staff  in  8 
nations 


Effects  Based  Assessment  (EBA) 


5  functions 

11  tasks 

15  staff  in  8 
nations 


R  functions 


5  functions 
5  tasks 

23  staff  in  8  nations 


To  EBA  To  and  from  EBA  From  EBA 

tit 
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Communication  Events 


High  level  documents 

-  Commander’s  initial 
guidance 

-  Campaign  Assessment 
Status  Report 

Generally  produced  by  EBP 
and  used  as  inputs  to  EBE, 
EBA 

Communication  events  were 
the  product  of  several  tasks 
and  triggered  another  set  of 
tasks 

Each  communication  event 
was  weighted  according  to  the 
scale  on  the  right 


•  Does  the  C.  E.  address: 

•  Completeness  -  Enemy/Friendly 

-  Who 

-  What 

-  When 

-  Where 

-  Why 

-  How 

•  Feasibility  of  resources 

-  Time 

-  Means 

•  Suitability 

-  Impact  on  mission  accomplishment 

-  Impact  on  compliance  with  commander’s 
guidance 

-  Impact  on  compliance  with  commander’s  intent 

•  Acceptable  risk 

-  Time,  equipment,  Soldiers,  position  losses 

•  Flexibility 

-  Enemy  COA1 

-  Enemy  COA2 

-  Enemy  COA3 


8 


Comments  on  Expected  a  priori  Model 

•  All  inputs  (with  exception  of  task  times)  obtained 
from  ConOps 

•  As  such,  the  expected  model  is  a  test  of  the 
ConOps 

•  ConOps  assigned  tasks  to  large  groups 
(planners)  vice  discrete  subsets 

•  Only  EBA  Chief  was  able  to  assign  specific 
tasks  to  individuals 

•  Task  times  were  estimated  based  on  pre-MNE  4 
process  rehearsal  and  subject  matter  expert 
advice 
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ORGANIZATION 


Model  - 


10 


EBP  Operator  Utilization  Rates 


%  Utilization 


EBP  Staff 


□  SusPlans 

□  OpLawPInr 

□  SOF  Plnr 

□  AstOpPInr 
BDepLandPInr 

■  KMOPIns 

■  DepSOFpInr 

□  DepAirpInr 

□  InOppInr 

□  Landplnr 

□  EBP  C 

□  IntelPInr 

■  MedPInr 

□  AirPInr 

□  OpSecPInr 

■  DStMobOf 

□  EBP  Dep 

□  CimicPInr  i 
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EBA  Operator  Utilization  Rates 


Operator  Utilization  Rates 
for  Other  Groups 

%  Utilization  Predicted 
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ORGANIZATION 


Communication  Outputs 

•  C3TRACE  default  decision  quality  setting 
used  (100%) 

•  Probability  of  making  a  good  decision  in 
expected  model  ranged  from  94%  to  1 00% 
based  on  the  average  information  quality 
of  communication  events 

•  Decision  quality  was  rated  as  “good” 

•  Cautious  interpretation  of  these  results 


Test 
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Data  Collected  in  MNE  4 


•  NASA  TLX  Workload  1 2  days 

•  Perception  of  Information  Quality 

•  Observations  of  task  completion 

-  Who  participated 

-  How  was  task  handled  (Large  group?  Smaller 
groups?) 

-  Task  time 

-  Communications  associated  with  tasks 

-  Delays  in  processing  task  and  explanations 

•  Observations  of  Communication  Events 

-  When/how  communicated  (email,  voice,  text, 
face) 

-  When  received  (delays?) 

•  Participant  comments  on  process  and 
organization 

-  Were  tasks  redundant  or  missing? 

-  Were  tasks  out  of  sequence? 


•  NASA  TLX: 

•  Mental  workload 

•  Physical  workload 

•  Time  Pressure  felt 

•  Satisfaction  with  performance 

•  Effort 

•  Frustration 

•  Information  Quality 

•  Accurate 

•  Appropriate 

•  Access 

•  Relevant 

•  Timely 

•  Complete 

•  Sufficient 

•  Concise 

•  Interpretable 

•  Understandable 


Data  Analysis 

•  Workload  varied  by  EBAO  group 

-  Overall  Means  for  each  group  will  be  used  to  adjust  operator 
utilization 

•  Information  quality  averaged  around  4. 2-4.6  for 

‘characteristic’  of  message  (accurate,  appropriate,  relevant,  timely)  but  slightly 
lower  (3. 8-3. 9)  for  ‘function’  of  message  (complete,  sufficient,  concise) 

•  Tasks  essentially  remain  unchanged  in  flow,  but  larger 
groups  broke  into  smaller  subgroups  for  efficiency 
(working  in  parallel) 

•  Task  times  were  fairly  close  to  actual  in  EBP  and  EBA 
but  not  EBE  (monitoring  operations  a  continuous  activity) 
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ORGANIZATION 


Model 


Refining  the  Model 

•  Task  assignments  will  be  readjusted  to 
reflect  MNE  4 

•  Information  quality  will  be  degraded  to 
match  actual  data 

•  Operator  utilization  will  be  modified  to 
match  reported  workload 

•  Refined  model  should  be  completed  by 
September  2006 
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AUTOMOTIVE  INFOCOM  TRANSPORT  &  AERONAUTICS  SPACE 

ENVIRONMENT 


DEFENCE  & 
SECURITY 


Developing  and  Comparing  HBR-Approaches 

continued 

Prof.  Dr.  Harald  Schaub 


TO  RTO  Task  Group  (SAS-053) 

A  Virtual  Institute  for  Research  on 
Human  Behaviour  Representation 

at  IDA  10-  12  Mai  2006 


Modelling  Requirements 


The  “five  commands”  of  HBR  modelling  are: 

1 .  Models  must  be  derived  from  real  life  phenomena,  e.  g.  typical  scenarios, 
real  events. 

2.  Models  must  have  construct  validity,  i.  e.  based  upon  scientific  constructs 
and  empirical  research  findings. 

3.  Models  must  have  face  validity  to  typical  representatives  of  the  real  life,  from 
which  the  models  were  derived. 

4.  Models  must  serve  non-trivial  analytical  and  training  purposes. 

5.  Models  must  be  open  to  validation  on  the  basis  of  real  life  data,  collected  by 
serious  scientific  research  (No  arbitrary  “data  farming”) 

6.  The  relevance  for  VI-HBR  is  to  have  first  and  decisive  rule  set,  which 
enables  the  user  to  evaluate  the  importance  of  modelling  approaches. 
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Scientific  Paradigms 


■  Mechanistic  engineers  minded 

■  Physical  approach 
Biological  approach 

■  Economic  resources  minded 
Computer  sciences  minded 
Systems  dynamics  approach 
Behavioural  and  social  sciences 
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Level  of  ‘behaviour’ 


■  Physiological 

■  (nerves  and  their  actions) 

■  Anthropometries  and  biomechanics 

(main  parts/members  of  the  human  body  and  the  manipulation  of  technical  parts) 
Individual  person 

(mental  actions,  actions  in  the  ‘outer  world’) 

Social  phenomena 

(group  dynamic  actions,  team  work,  inter-personal  actions) 

■  Organisational  phenomena 

(interaction  between  organisations,  individuals  and  groups) 

Inter-cultural  phenomena 

(interaction  between  people  with  highly  different  cultural  backgrounds) 
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General  model  of  action  organisation 


The  phases/different  processes  of  information  processing  of  a  General  Model 
are 

goal  elaboration 

■  information  gathering 
mental  modelling 

■  prognosis 
planning 
decision  making 
acting 

■  effect  control 

■  (self)  reflexion 

(self)  modification/adaptation. 
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General  model  of  action  organisation 


The  action  levels  range  from 

■  Reflex 

(e.  g.  eyelid  twitch) 

Automatism  /  skill  based  behaviour 
(e.  g.  acting  in  panic) 

■  Trail-and-Error 

■  Routine  /  rule  based  behaviour 

(e.  g.  routine  manipulations  of  engines,  like  driving  a  car) 
Problem-Solving  /  Knowledge  based  behaviour. 
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ORGANIZATION 


Rassmusen  Ladder 


Goals 


Sensory  Input  Signals  Actions 
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Phenomena/Factors  on  the  task  level 


■  External  task-definition 

■  Attended  time 

■  Efficiency  of  agents 
Efficiency  of  group 

■  Influence  HK 
Epistemic  competence 
Compiling  agents 

■  Obstacle 
Expected  efficiency 
Heuristic  competence 
Competence/Expertise 
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■  Complexity 

■  Agents'  Coordination 

■  External  coordination 

■  Agents'  Motivation 

■  requirement/competenc 
e  fitting 
Indefiniteness 

■  Variability 

■  Time  frame/available 
time 

Perceived  efficiency 

■  Time  required 

■  Pressure  of  time 
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Phenomena/Factors  on  the  individual  level 


Intentions 

General  education 

Coverage  /  Accomplishment  of  fears 

Mastery  of  tasks 

Education  and  further  education 

Balance  /  Stability 

Automatism 

Mobility  /  Flexibility 

"Combined-Experiences" 

Ability  to  delegate  /  Ability  to  depute 

Empathy 

Decisions 

Decision-making  ability 
Resoluteness 
"Epistemic  Expertise" 

Pool  of  experience 
Experience  concerning  success 
Confidence  concerning  success 
Ethics  and  moral 
Ethnological  Experience 
Know-how  /  expertise 
Coping  with  frustration 
Leading-qualities  /  Talent 
Health 
Basic  skills 
Actions 

"Heuristic  competence" 
Cooperativeness 
Hypotheses  and  models 
Informations 
Intelligence 

Intercultural  decision  making 
"Joint-experiences" 


Comradeship 

"Knowledge  of  organisational 
structure" 

Cognitive  capacities 
Ability  to  communicate 
Acceptance  of  complexity 
Stamina 

Ability  to  cope  with  conflicts 
Control 

Willingness  to  cooperate 
Communication  skills 
Fitness 

Power  /  Strength 
Creativity 

Applied  geography 

Careers 

Productivity 

Learning  aptitude 

Shortcoming  /  Insuffiency  /  Flaw 

Knowledge  of  human  nature 

Military  skills 

"Ability  to  moderate  and  mediate" 

Ability  to  motivate 

Motives 

Curiosity 

Individual  characteristics 
Plans 

Problem  solving 
Forecast  and  expectations 
Emotional  stability 
Legal  expertise 
Reflexes 

Reflection  and  modification 


Ability  to  regenerate 

Regulation  levels 

Religion 

Risk  awareness 

Routine 

Threshold  of  pain 
Education 
Self-confidence 
Desire  to  win 
Soldierly  behaviour 
Language  skills 
Knowledge  of  strategies 
Coping  with  stress 
Knowledge  of  tactics 
Capacity  for  teamwork 
Persuasiveness 
Exposure  to  pressure  of  time 
Tolerance  of  Indefiniteness 
Impartiality 

Sense  of  responsibility 
"Trial  and  error" 

Reliability 

Perception 

Appreciation 

Knowledge  and  cognitive  skills 
Targets 
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Phenomena/Factors  on  the  group/team  level 


■  CPS  scheme 
Extent  of  hierarchy 

■  Fitting  of  leadership 
Formality  of  interactions 

■  "Groupthink." 

Impact  of  leadership 

■  Isolation 

■  Presence  of  the  leader 
Scheme  of  cooperation 

■  Ways  of  leadership 
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Phenomena/Factors  on  the  organization/interagency  level 


Within  organisations 

■  Analytical  division  of  labour 

■  Number  of  groups 

■  Eye  level  contact 

■  CMC 

■  Degree  of  differentiation 

■  Deployment 

■  Size  of  groups 

■  Command  structure/hierarchy 

■  Information  management 

■  Knowledge  of  Corporate  Culture 

■  Cohesion 

■  Organisational  division  of  labour 

■  Corporate  Culture 

■  Part  -  organisations 
Virtual  elements 


Between  organisations 

■  Analytical  work  sharing  between 
organizations 

■  Divergence  of  organization 
cultures 

■  Hierarchy 

Inter-organisationalen  contacts  on 
an  equal  level 
Inter-organizational  goals 

■  Inter-organizational  information 
management 

Knowledge  about  shared  values 
and  norms 

Number  of  different  reporting 
channels 

Organizational  work  sharing 
between  organisations 
"Virtual  inter-organizational 
elements" 

■  Ways  of  communication 

■  Ways  of  cooperation 
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General  model  with  some  interactions 


Complexity 

Ability  to  work  in  Teams 

Dynamics 

Intercultural  Competencies 

Threat 

Mental  Stability 

Many  Goals 

Leadership 

Uncertainty 

Military  Abilities 

Knowledge 

Irreversibility 

Physical  Fitness 

Intercultural  Environment 

Self-Efficacy 

Deprivation 

Factors  of  the  Person 

Factors  of  the  Situation 

Individual  Problem-Solving 


Group  Context 


Leadership 


Organizational  Context 


Inter-Organizational  Cooperation 


Society 


Intention  Selection  |<~> 

Goal  Elaboration  |<-> 

Information  Gathering  |<-> 

Mental  Modeling  \<r> 

Prognosis  |<-> 

l<-» 


Planning 


Decision  Making  |<-> 

Action  |<-> 

Effect  Control  |<-> 

(Self-)  Reflection  ]<-> 

Self-Modification  |<-> 


Reflexes 


Automatisms 


Routines 


Tasks 


Problems 


Trial  and  Error 


International  Relations 
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Comparing  (general)  Approaches 


Agent  Effectors 


Agent  Function 

~T 


Environment 


General  Model 


Perception 
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(Some)  of  the  requirements  for  the  HBR  virtual  institute  (from  „Final  Report 
from  ET.V  for  SAS-053  approval”): 


Develop  a  virtual  institute  technology  development  strategy; 

Develop  virtual  institute  metrics  and  success  criteria; 

■  Develop  a  virtual  institute  research  plan  and  strategy; 

Determine  the  key  steps  for  developing  the  virtual  institute  and  the  key  technologies  needed  by  the  virtual  institute 
to  conduct  initial  experiments; 

■  Develop  a  methodology  to  assess  technical  benefits  of  virtual  institute-based  experiments  and  tests  as  compared  to 
conducting  equivalent  experiments  and  tests  in  one  location; 

■  Develop  descriptions  of  partitions  of  work  between  national  participants  based  on  capabilities  and  national  interests; 
Develop  guidelines  for  technology  sharing  among  virtual  institute  participants; 

Identify  and  prioritize  experiments  and  tests  that  are  of  common  interest  to  the  participating  countries; 

Develop  ontologies  for  human  behaviors; 

Develop  and  employ  a  methodology  that  can  be  used  to  organize  a  human  behavior  model  library  for  each 
country  and  for  all  countries  as  well  as  enable  the  development  of  common  human  behavior  models  and 
model  components; 

■  Develop  a  virtual  institute  human  behavior  model  experiment  plan; 

Determine  common  behaviors  and  country  specific  behaviors  via  experiments; 

■  Determine  commander  behavior  modeling  techniques; 

■  Develop  a  sequence  of  experiments  and  tests  that  will  demonstrate  benefits  and  desired  level  of  international 
cooperation  for  virtual  institute  human  behavior  model  development 

■  Provide  capabilities  for  serving  as  a  clearing  house  for  research  results 

■  Provide  a  library  of  research  results,  ongoing  academic  research,  and  experiments 

■  Insure  that  HBR  studies  are  coordinated  between  nations 

■  Serve  as  a  testbed  host  for  distributed  experiments 

■  Distribute  human  performance  models  to  organizations  that  need  them 

■  Serve  to  foster  cross-national  academic  research  support  for  HBR  model  development  and  support  validation  of 
models 

Development/promulgation  of  standards  or  measures  for  human  behavior  models,  developing  extensible 
standards  for  describing/defining  human  behavior  models,  and  determining  when  the 
conditions/circumstances  where  each  model  applies 

■  Development  of  composable  HBR  data  models  and  development  of  the  means  for  storing  data  in  a  standard 
manner  so  that  it  can  be  reused  across  experiments 
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Your  Contact 


Prof.  Dr.  Harald  Schaub 
Human  Factors  Team 
IABG  mbH  -  VG  63 

EinsteinstraRe  20 
85521  Ottobrunn 
Fon  +49  89  6088-3178 

Fax  +49  89  6088-133178 

schaub@iabq.de 
humanfactors@iabq.de 

www.iabq.de 
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Chapter  4  -  VI  CONCEPT 


4.1  INTRODUCTION 

The  idea  of  virtual  research  organizations  is  driven  by  economic  and  security  requirements  on  one  hand 
and  by  technological  progress  especially  in  high-speed  digital  communications  and  the  semantic  web  on 
the  other.  Advances  in  technology  play  a  major  role  for  economic  growth  and  security  in  the  information 
age  we  are  now  in.  Research  and  Development  (R&D)  is  an  important  area,  which  can’t  be  neglected. 
Research  institutes  and  how  they  work  together  play  a  major  role  in  this  regard.  The  capability  and 
efficiency  of  research  institutes  working  in  an  international  network  is  therefore  important.  Today’s  high 
technology  research  is  furthermore  interdisciplinary  and  requires  cooperation  of  many  different  research 
areas,  which  normally  can’t  be  found  located  in  one  place.  The  exchange  of  information  has  to  keep  pace 
with  the  accelerating  rate  of  change  of  technology  development  and  can’t  relay  on  the  traditional  means  of 
publications,  conferences  and  workshops  any  more.  On  the  other  hand  information  technology  has 
changed  ways  of  communication  and  opened  a  global  information  network.  The  use  of  this  network  is 
more  and  more  common  and  the  younger  generation  is  used  to  it  in  their  private  live.  They  expect  this 
kind  of  communication  and  information  exchange  also  in  their  business  or  research  life. 

NASA  with  their  Virtual  Research  Center,  which  started  in  1995,  was  one  of  the  first,  building  a  virtual 
research  organization.  Others  followed  and  the  European  Union  started  in  2000  with  a  program  supporting 
the  build  up  of  virtual  institutes  to  enable  a  better  cooperation  of  research  institutes  and  the  industry. 
NATO  RTO  started  to  discuss  this  topic  also  in  the  year  2000.  The  concept  of  a  Virtual  Institute  or 
Laboratory  was  introduced  by  Rene  Willems  (SAS  Panel  Chair  at  that  time)  in  the  RTB  Meeting  in 
September  2000  to  NATO.  The  LTSS  on  Human  Behaviour  Representation  Study  Group  discussed  this 
topic  also  in  their  meetings  and  recommended  a  follow-on  study  on  the  Virtual  Institute  Concept  in  their 
Final  Report.  The  SAS  Panel  approved  this  recommendation  and  established  an  Exploratory  Team  (ET.V), 
which  developed  a  program  of  work  for  the  research  study  group  SAS-053. 

As  said  above  NATO  R&D  depends  on  the  cooperation  of  the  different  research  entities  in  NATO 
(RTO  with  its  Panels  and  RTGs,  NC3A,  NURC),  in  international  and  national  scientific,  governmental 
and  non-governmental  organizations  and  in  industry.  All  these  entities  could  be  users/members  of  a  virtual 
research  organization. 

Potential  advantages  of  the  use  of  virtual  institutes  are: 

•  Costs/Economics: 

•  Reduce  of  travel  time  and  cost; 

•  Reduce  cost  per  experiment/exercise;  and 

•  Reduce  elapsed  time  between  experiment/exercise  conception  and  execution; 

•  Cooperation: 

•  Reducing  duplication  of  effort; 

•  No  limitation  of  size  and  scale  of  effort  by  one  institute  or  even  one  nation; 

•  Improve  re-use/exploitation  of  prior  work; 

•  Foster  inter  RTO  Panel  research;  and 

•  Improve  access  to  international  expertise. 
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•  Quality: 

•  Better  peer  review  of  results;  and 

•  Bringing  in  different  regional/cultural  knowledge. 

•  Technology  and  Opportunities: 

•  Faster  research  distribution;  and 

•  Experience  from  other  studies  from  internet. 

•  Security: 

•  Better  control  of  access  to  critical  information;  and 

•  Better  protection  of  secured  areas  because  of  limited  physical  access. 

In  the  following  a  definition  for  a  virtual  institute  is  given.  Underlying  technologies  and  their  development 
and  use  for  the  virtual  institute  are  discussed  in  the  next  section.  Besides  technologies  and  their  use  the 
organization  of  a  Virtual  Institute  plays  a  major  role.  A  recommended  approach  is  shown  in  Section  4.4. 
Section  4.5  deals  with  the  usage  of  a  Virtual  Institute  in  RTO  and  gives  practical  examples.  Security  and 
intellectual  property  will  have  an  impact  on  data  exchange  in  a  NATO  RTO  Virtual  Institute.  Section  4.6 
highlights  some  issues  on  this  topic.  In  Section  4.7  you  will  find  a  short  description  of  the  RTO  website. 

4.2  DEFINITION 

The  European  Commission  (EC)  defined  a  Virtual  Institute  in  a  press  release  regarding  their  Fifth 
Framework  Programme  as:  “A  Virtual  Institute  links  geographically  scattered,  complimentary  research 
and  industrial  elements  in  order  to  transfer  and  implement  research  results  rapidly,  primarily  in  industrial 
applications  and  services.  It  can  be  thought  of  as  a  market-oriented  network”  (see  [2]).  The  intention  of  the 
EC  was  to  establish  Virtual  Institutes  as  independent  and  self-financing  legal  entity,  which  could  provide 
services  to  European  industry,  government  and  research  organizations. 

In  the  research  area  the  concept  of  Virtual  Laboratories  was  discussed  starting  in  1996.  The  term 
collaboratory  was  coined  when  William  Wulf  merged  “collaboration”  and  “laboratory”  to  describe  a 
“...  center  without  walls,  in  which  the  nation’s  researchers  can  perform  their  research  without  regard  to 
geographical  location  -  interacting  with  colleagues,  accessing  instrumentation,  sharing  data  and 
computational  resources,  and  accessing  information  in  digital  libraries”  (see  [4]).  In  1999  an  Expert  Meeting 
on  Virtual  Laboratories  was  held  at  the  International  Institute  of  Theoretical  and  Applied  Physics  (IITAP), 
IOWA  State  University  sponsored  by  the  United  Nations  Educational,  Scientific  and  Cultural  Organization 
(UNESCO)  (see  [3]).  This  meeting  came  up  with  following  definition  of  a  Virtual  Laboratory: 
“An  electronic  workspace  for  distance  collaboration  and  experimentation  in  research  or  other  creative 
activity,  to  generate  and  deliver  results  using  distributed  information  and  communication  technologies”. 
A  major  point  in  this  definition  is  that  people  at  various  sites  could  work  together  collaboratively  and 
simultaneously  having  access  to  a  digital  library.  The  experts  at  the  meeting  saw  three  main  directions  for  a 
Virtual  Laboratory: 

•  Project  driven  with  a  well  defined  goal  and  a  fixed  timescale; 

•  Large  and  complex  projects  with  interdisciplinary,  cross-organizational  participation  and  the  sharing 
of  major  experimental  devices;  and 

•  More  general  interactions  among  members  of  a  common  specialist  community  with  multi-national 
collaboration. 

The  Exploratory  Team  (ET.V)  discussed  in  its  final  report  the  topic  why  to  call  it  a  Virtual  Institute  and 
not  Laboratory,  Testbed  or  Workshop.  Main  reason  for  that  was  that  an  institute  is  a  much  wider  term, 
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which  connects  very  well  with  its  research  orientation.  A  laboratory,  testbed  or  workshop  is  normally  a 
part  of  an  institute  and  sometimes  even  has  a  very  limited  lifespan  like  a  testbed  or  a  Virtual  Institute  as 
defined  above.  Other  parts  that  could  be  hosted  by  an  institute  include  a  (digital)  library,  a  web-based 
clearinghouse,  an  education  and  training  facility  (including  e-leaming)  and  other  digital  services.  Our 
definition  of  a  Virtual  Institute  is: 

“A  Virtual  Institute  is  a  permanent  collaborative  research  environment  with  its  own 
organization  providing  support,  facilities  and  tools  for  research,  experiments,  collaboration, 
and  distribution  of  research  results.” 

An  important  point  in  our  view  is  that  the  Virtual  Institute  is  a  permanent  entity  which  has  its  own 
attached  organization  not  only  to  run  and  maintain  all  technical  facilities  and  keep  it  running  but  also  to 
steer  the  institute  and  its  development  regarding  the  selection  of  future  research  projects,  experiments  and 
the  further  development  of  collaboration  facilities  and  services  like  standardisation,  training  and  education 
and  legal  issues  as  intellectual  property  and  security. 


4.3  UNDERLYING  TECHNOLOGY 

Core  technology  to  realize  a  Virtual  Institute  as  a  collaborative  research  environment  is  Information 
Technology  (IT).  IT  is  a  general  term  that  describes  any  technology  that  helps  to  produce,  manipulate,  store, 
communicate,  and/or  disseminate  information.  For  the  Virtual  Institute  concept  the  communication  and 
dissemination  aspect  plays  a  major  role  to  link  computer  systems,  research  tools  and  other  communication 
equipment  like  video  or  voice  in  multiple,  geographically  distributed  locations.  This  chapter  will  give  an 
overview  on  current  developments  and  available  tools  in  this  area.  A  more  detailed  view  can  be  found  in  [6]. 

J.  Vary  distinguishes  between  three  classes  of  communication  in  his  taxonomy  of  Virtual  Laboratory  tools 

[3]: 

•  Person-to-person  communication  in  a  network  of  scientists; 

•  Person-to-equipment  communication  to  control  a  network  of  tools;  and 

•  Person-to-metamachine  communication  which  involves  access  to  an  intelligent  network  of 
information  and  computing  resources. 

For  all  classes  of  communications  the  transfer  of  digital  media  over  a  common  network  is  essential. 
Digital  media  includes: 

•  Text  (formatted  and  unformatted); 

•  Graphics  (pixel  or  vector  oriented); 

•  Audio; 

•  Video;  and 

•  Computer  programs. 

The  kind  of  media  and  the  amount  determine  the  necessary  bandwidth  for  operation.  In  Europe  and  North 
America  is  in  most  places  a  sufficient  Internet  connection  available.  If  in  the  case  of  experiments  or  other 
special  projects  involving  subject  matter  experts  or  scientists  from  other  nations  a  connection  to  other 
places  has  to  be  established  (e.g.  to  an  ongoing  operation  like  ISAF)  special  lines  might  have  to  be  build 
up  or  rented.  Also  if  secure  lines  are  needed  it  has  to  be  checked  if  the  desired  connectivity  is  available. 

The  technologies  underpinnings  needed  to  implement  a  virtual  institute  have  only  recently  become 
available  at  a  cost  and  performance  that  makes  their  use  efficient  and  effective  for  all  NATO  members. 
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4.3.1  Person-to-Person  Communication 

This  class  includes  all  conventional  forms  of  human  interaction  such  as  conversation,  telephone  call,  book 
or  report,  letter,  or  TV  and  their  computer- supported  analogs  voice  over  IP,  video  teleconference  (VTC), 
e-mail  and  the  WWW.  Table  4-1  shows  the  division  in  synchronous  and  asynchronous  communications. 

Table  4-1 :  Person-to-Person  Communication 


Synchronous 

Asynchronous 

Telephony 

Chat 

Whiteboard 

Video  teleconference 

Virtual  awareness 

Application  sharing 

Shared  workspace 

E-mail 

File  exchange 

Internet  radio 

Computer  Supported  Cooperative  Work 
WWW 

Following  tools  are  widely  available: 

•  Telephone 

The  classic  analogue  phone  is  more  and  more  replaced  by  voice  over  IP  systems.  Any  computer 
who  is  linked  to  the  Internet  can  be  used  for  phone  calls  or  even  audio  conferences.  The  quality  of 
this  service  over  the  open  Internet  is  not  always  high.  There  is  also  special  equipment  available 
for  conference  systems. 

•  Chat 

Chat  is  a  text  based  synchronous  communication  over  a  computer  network  (e.g.  the  Internet). 
Its  demands  on  bandwidth  and  quality  of  service  are  very  low.  There  are  lots  of  chat-servers 
installed  on  the  Internet.  Chat  software  comes  normally  also  with  office  packages  like  Microsoft 
Office  and  allows  closed  user  groups  in  an  office  environment. 

•  Whiteboard 

A  Whiteboard  extends  the  possibilities  of  a  pure  text  chat  system  to  the  exchange  of  any  graphical 
information.  Special  systems  like  for  mathematical  equations  are  available.  A  Whiteboard  system 
could  also  be  used  to  share  the  output  of  a  computer  program  in  real-time  on  the  screen  with  a 
remote  user. 

•  Video  Teleconference  (VTC) 

VTC  can  run  on  dedicated  lines,  ISDN  lines  or  over  the  Internet  and  is  not  very  expensive  any  more. 
Dedicated  lines  give  a  better  quality  than  the  Internet  but  Internet  VTC  is  usable.  For  Internet  VTC  a 
special  server  is  necessary  which  the  Virtual  Institute  could  host  or  it  could  be  an  available  server 
like  from  Apple  me.com  or  AOL  AIM,  etc.,  with  a  low  fee.  Very  often  VTC  applications  offer  also 
other  functions  like  whiteboard  or  chat. 

•  Virtual  Awareness 

Behind  virtual  awareness  is  a  concept  of  virtual  vicinity.  Web  users  are  able  to  sea  each  other  when 
serving  the  same  web  site  or  any  other  kind  of  vicinity  defined  in  the  respective  server.  This  would 
allow  starting  a  synchronous  communication  with  each  other.  This  concept  could  be  used  on  the 
Virtual  Institute  server  to  facilitate  cooperation. 

•  Application  Sharing 

Systems  like  Windows  Terminal  Server  or  MAC  OS  X  screen  sharing  allow  users  to  log  into  a 
system  remotely.  This  could  be  very  useful  during  a  VTC  or  an  experiment.  Also  for  giving  support 
to  other  users  it  could  be  very  helpful. 
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•  Shared  Workspace 

Shared  workspace  could  be  a  place  to  store  and  share  files  like  in  Microsoft’s  Shared  Workspace 
or  a  system  that  offers  a  complete  office  environment  with  virtual  conference  rooms  and  audio 
support  like  Ezenial’s  Info  Workspace. 

•  E-mail 

Electronic  mail  allows  the  exchange  of  text  and  any  electronic  files  as  attachments.  A  user  is 
connected  with  his  e-mail  client  to  a  mail  server.  The  information  is  passed  between  different  mail 
servers  and  can  at  the  end  being  downloaded  from  the  server.  E-mail  client  can  be  any  computer 
connected  to  the  network  or  even  mobile  devices  (e.g.  iPhone  or  BlackBerry).  The  amount  of  data 
that  can  be  transferred  in  an  e-mail  is  limited  by  the  server.  Typical  limits  are  10  MB  per  e-mail. 

•  File  Exchange 

Files  can  be  exchanged  as  said  as  attachments  to  e-mails,  but  normally  the  amount  of  data  is  pretty 
small.  To  exchange  files  of  any  size  over  the  Internet  ftp  (file  transfer  protocol)  is  used.  The  file 
must  be  stored  on  a  server,  which  supports  the  ftp  protocol.  The  user  who  wants  to  download  the 
file  from  his  computer  has  to  know  the  address  (usually  as  URL)  and  must  have  access  rights  to 
the  server.  For  the  transfer  a  normal  web  browser  can  be  used.  Some  ftp  programs  are  available, 
which  offer  a  better  user  interface. 

•  Internet  Radio 

The  Internet  offers  the  possibility  to  broadcast  audio  like  a  radio  station.  With  a  special  client  like 
iTunes  or  web  browsers  the  broadcast  can  be  received. 

•  Computer  Supported  Cooperative  Work  (CSCW) 

CSCW  tries  to  coordinate  the  work  of  a  team  of  persons  on  one  document  or  task  by  providing 
version  control,  a  comment  process,  access  rights  management,  archiving,  etc. 

•  WWW 

The  World  Wide  Web  is  a  distributed  hypertext  system.  The  information  is  stored  on  servers  and 
referenced  by  hyperlinks.  Multi-media  content  such  as  pictures  or  videos  is  embedded  in  the 
hyperlink  mechanism  and  can  be  displayed  in  web  browsers.  Programs  like  Java  applets  can  be 
embedded  in  the  hyperlink  documents  and  executed  on  the  client  system  after  download. 

In  the  last  years  the  web  got  very  popular  and  is  now  widely  used  in  business  and  social  life 
especially  by  the  younger  generation.  Web  2.0  is  a  term  describing  the  trend  to  more  collaboration 
between  the  web  users.  This  is  made  possible  by  better  and  more  intuitive  user  interfaces,  easier 
web  design  and  tools  to  build  own  web  pages  by  inexperienced  users,  decreasing  cost  for  internet 
access  and  hosting  own  web  sites  and  community  sites  like  MySpace.  Also  the  application 
interfaces  of  web  applications  can  now  be  seamlessly  integrated  in  other  applications  (e.g.  Google 
Search). 

4.3.2  Person-to-Equipment  Communication 

Experiments  play  a  major  role  in  research.  It  can  be  physical  experiments  using  probes,  manipulators  and 
measuring  devices  or  simulators  and  simulations  run  on  a  computer  system.  Very  often  the  need  for 
remote  control  exists.  This  could  be  done  online  (teleoperation)  or  by  executing  a  prepared  script 
(teleprogramming).  Sometimes  a  mix  of  both  methods  is  applied. 

Besides  application  sharing  (see  above)  in  most  cases  special  tools  are  used. 

4.3.3  Person-to-Metamachine  Communication 

The  concept  of  person-to-metamachine  communication  deals  with  the  analysis  of  huge  amounts  of 
distributed  data  in  a  computer  network  with  intelligent  human  computer  interfaces.  Prof.  John  Markoff 
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coined  the  term  “Web  3.0”  in  the  New  York  Times  2006  to  describe  the  next  step  in  the  development  of 
the  www.  Technologies  engaged  are: 

•  Intelligent  applications  (natural  language  processing,  machine  learning,  machine  reasoning, 
autonomous  agents); 

•  Distributed  databases  and  database  integration  (semantic  web);  and 

•  Network  or  distributed  computing. 

The  first  steps  in  this  develop  have  been  done  with  client-server  computing,  remote  database  clients  and 
agent  technology,  but  it  is  still  a  long  way  to  come. 

The  Wiki  as  software  that  allows  a  user  community  to  edit  web  pages  interactively  is  one  of  the  first 
applications,  which  could  be  seen  as  Web  3.0. 


4.4  ORGANIZATION  AND  BUILD  UP  OF  A  VIRTUAL  INSTITUTE 

Wikipedia  defines  institute  as:  “An  institute  is  a  permanent  organizational  body  created  for  a  certain 
purpose.  Often  it  is  a  research  organization  (research  institution)  created  to  do  research  on  specific  topics.” 
A  Virtual  Institute  in  the  RTO  should  be  a  permanent  institution  under  which  specific  research  projects 
with  a  fixed  time  scale  are  executed. 

The  VI  needs  to  have  an  in-depth  knowledge  of  its  subject  and  is  likely  to  involve  some  of  the  leading 
experts  in  its  research  area.  It  must  also  have  a  finely  tuned  internal  and  external  communication  system 
based  on  a  genuine  thrust  and  a  spirit  of  co-operation  between  its  members  and  indeed  with  its  customers. 
A  permanent  VI  staff  could  provide  the  corporate  knowledge  regarding  the  specific  research  area  which 
the  Virtual  Institute  represents  (e.g.  on  Human  Behaviour  representation),  Virtual  Research  issues, 
maintain  critical  services,  work  on  standards  and  their  application  to  be  used  in  this  context  and  give 
support  to  questions  regarding  security  and  intellectual  property  issues.  This  way  an  in  NATO  R&D 
needed  capability  could  be  established  that  supports  the  rapid  formation  of  ad  hoc,  multi-disciplinary 
research  teams,  accelerates  their  work  by  making  prior  work  and  related  expertise  readily  available,  fosters 
rapid  dissemination  of  research  results,  and  otherwise  aids  in  the  execution  and  exploitation  of  relevant 
research. 

Figure  4-1  shows  a  recommended  Virtual  Institute  Organization  with  permanent  stuff  elements  and 
temporary  projects,  which  will  be  executed  under  the  virtual  institute.  The  main  structure  of  the  virtual 
institute,  such  as  operational  executive,  administration,  IT  support  and  VI  support  would  be  relatively 
fixed  whereas  specific  projects  and  experiments  with  their  national  participation  could  vary. 
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Figure  4-1 :  Virtual  Institute  Organization. 

The  operational  executive,  the  administration  and  the  multi-national  control  board  provide  guidance  and 
oversight  to  project  teams,  aid  in  the  formation  of  virtual  teams  and  in  the  coordination  between  them  and 
the  VI  permanent  stuff.  They  are  also  responsible  for  the  operation  and  further  development  of  the  virtual 
institute.  The  virtual  institute  should  continually  evaluate  its  work,  improve  organization  and  procedures 
and  publish  guidance  for  other  virtual  institutes. 

The  other  permanent  stuff  elements  are  the  IT  support  and  the  research  support  stuff. 

The  IT  support  is  responsible  for  ongoing  technical  services  for  all  stuff  members  and  the  project  teams. 
It  aids  in  making  the  technical  connectivity  needed  by  the  virtual  teams  in  order  for  them  to  exist  and  also 
provide  data  storage,  access,  protection,  data  mining  capabilities  and  security  and  privacy  services  for  the 
institute’s  data.  The  IT  support  will  assist  the  virtual  teams  in  placing  their  raw  and  analyzed  data  from 
experiments  and  research  into  permanent  storage.  IT  support  will  also  assist  all  users  with  education  and 
training  on  the  IT  facilities  and  how  to  use  them. 

The  research  support  stuff  has  two  functional  areas:  the  direct  support  of  the  project  teams  in  their  area  of 
expertise  and  the  further  development  of  the  virtual  institute  regarding  the  research  issue  at  hand  and 
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procedures  and  management.  Direct  support  based  on  experience  can  be  given  to  the  project  teams  in 
helping  with  setting  up  experiments,  helping  with  the  evaluation  of  experiments,  guiding  project  teams 
through  the  process  of  working  under  a  virtual  institute,  and  helping  with  data  exchange  issues  regarding 
intellectual  property  rights  and  security.  Each  project  team  should  have  one  permanent  point  of  contact  in 
the  research  stuff  who  might  be  a  full  member  of  the  project  team.  The  research  support  stuff  will  be 
responsible  for  the  function  of  the  virtual  institute  to  serve  as  a  clearinghouse  by  organizing  the  peer 
review  of  research  results  from  the  project  teams  and  for  the  proper  storage  with  references  (for  future  use 
on  the  semantic  web).  Ideally  the  research  stuff  would  organize  the  digital  library  of  the  institute  in  a  way 
that  all  relevant  information  on  the  institute’s  subject  can  be  found  on  their  server  or  by  crosslink  to  other 
relevant  organizations.  The  research  support  could  also  identify  research  and  development  needs  and  gaps 
in  the  particular  field  of  expertise  and  document  the  state  of  the  art  in  the  research  area  of  the  virtual 
institute  and  participate  in  the  discussion  of  a  future  roadmap  of  this  research  area.  Research  support 
should  have  also  a  part  attached  which  is  responsible  for  the  future  development  of  the  virtual  institute 
concept  by  evaluation  of  ongoing  work  and  work  on  process  development  and  standards. 

The  project  teams  would  be  built  for  specific  research  project  with  a  limited  lifetime.  Each  project  would 
have  a  well-defined  research  objective  and  when  that  objective  is  achieved  the  team  can  disband.  They 
would  consist  of  members  from  different  participating  nations  and  NATO  HQs  and  agencies. 

The  build  up  of  a  Virtual  Institute  includes  three  work  packages  (see  [9]): 

•  Establishment  and  Management  with  the  main  objective  to  define  and  establish  the  VI  organization 
and  management  routines,  consisting  of: 

•  Legal  establishment;  and 

•  Management  and  administration. 

This  includes  the  setup  and  definition  of  the  virtual  institute’s  research  issue  and  has  therefore 
been  done  on  RTO  level. 

•  Infrastructure  with  the  main  objective  to  establish  an  effective  infrastructure  supporting  both 
internal  and  external  cooperation,  consisting  of: 

•  Concept  and  specifications  of  the  infrastructure; 

•  Technical  solutions  and  associated  costs; 

•  Development  of  the  infrastructure; 

•  Deployment  and  validation; 

•  Training  and  operation;  and 

•  External  users’  portal. 

A  major  point  of  this  step  is  the  decision  on  the  infrastructure  for  the  virtual  institute.  This  includes 
not  only  IT  systems  but  also  facilities  to  host  the  stuff  and  all  technical  infrastructure.  Another 
important  point  is  the  question  regarding  the  financing  of  the  institute.  As  explained  above  the 
virtual  institute  needs  permanent  stuff  elements  to  function  properly. 

•  Management  and  sharing  of  knowledge  with  the  main  objective  to  secure  corporate  knowledge  - 
and  especially  ensure  corporate  memory,  consisting  of: 

•  Web-based  research  result  and  project  database; 

•  Identification  of  profiles  and  experts; 

•  Design  of  knowledge  centre;  and 

•  Development  of  methods  and  routines. 

This  package  is  an  ongoing  process,  which  will  be  the  responsibility  of  the  research  stuff. 
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Additionally  to  the  build-up  of  the  institute  each  of  the  projects  to  be  performed  under  the  virtual  institute 
has  to  be  planned  and  set  up  appropriately.  Social  and  psychological  factors  have  to  be  considered. 
So  should  the  project  especially  when  set  up  with  experts  from  different  regions  and  cultural  backgrounds 
start  with  face-to-face  meetings  to  know  each  other  better  before  working  in  a  virtual  environment.  The  set 
up  of  the  virtual  environment  must  also  reflect  different  personalities  in  the  project  and  requirements 
coming  from  the  research  topic. 

4.5  USAGE 

This  section  will  discuss  the  use  of  virtual  institute  technology  in  the  RTO  with  reference  to  already  existing 
capabilities  like  the  RTO  forum  with  the  WISE  information  panel,  the  NATO  Combined  Federated  Battle 
Lab  Network  and  NATO  Science,  Technology  and  Research  Network  (STARNET). 

Overall  objective  of  the  virtual  institute  is  to  provide  services  to  the  research  community  in  the  RTO  as  a 
whole  and  to  special  project  teams  in  particular  with  direct  support  of  them  providing  a  virtual  environment. 
The  virtual  institute  could  deliver  following  products  to  support  study  teams  in  working  together  (discussion 
forum,  message  board,  etc.),  to  publish  research  results  (digital  library),  to  give  advice  to  study  teams 
(SME  contact  data,  legal  guidance,  checklists)  and  to  conduct  experiments  (model  library,  data  and 
scenarios,  raw  data  for  analysis): 

•  Digital  Library  or  Information  Resource  Centre 

The  digital  library  could  consist  of  research  documents,  project  documentation,  simulation  models 
including  their  documentation  and  scenarios,  links  to  external  relevant  sources.  The  RTO  web  site 
offers  already  some  of  this  functionality  with  access  to  RTO  documents  and  workspaces  for 
specific  projects.  No  model  library  for  any  research  area  is  available.  Also  no  data  on  scenarios 
used  for  research  or  other  purposes  (e.g.  training  and  exercises)  is  available  on  RTO  web. 

•  Taxonomy 

An  overall  research  area  or  comprehensive  taxonomy  could  be  provided.  RTO  developed  together 
with  ACT  a  cross  referencing  mechanism  based  upon  a  common  technology  taxonomy  (UK  MOD), 
which  is  provided  to  each  workspace  on  the  RTO  web. 

•  Discussion  Forum 

A  discussion  forum  can  be  used  project  internal  or  with  a  wider  community  of  users  to  discuss 
current  issues.  It  needs  always  moderation.  In  the  RTO  collaborative  environment  a  forum  for  each 
workspace  is  established. 

•  Message  Board  or  News 

A  message  board  or  news  allows  to  post  messages  for  a  specified  user  community.  This 
functionality  is  on  the  RTO  web  available. 

•  Shared  Calendars 

Shared  calendars  allow  to  place  events  and  to  coordinate  scheduling  in  a  team.  The  functionality 
of  shared  calendars  can  be  extended  to  a  management  of  common  resources  like  conference 
rooms  or  VTC.  On  the  RTO  web  are  the  dates  of  RTO  events  placed  but  on  the  project  level  no 
shared  calendars  for  the  teams  exist. 

•  Virtual  Conferences 

The  virtual  institute  can  organize  real  or  virtual  conferences  to  specific  topics  of  its  research  area  or 
support  the  set  up  of  virtual  conferences  for  project  teams.  Live  conferences  could  be  broadcasted 
via  TCP/IP  or  recorded  and  then  podcasted.  This  is  not  available  at  the  moment. 

•  Lecture  Series 

Lecture  series  could  be  supported  by  a  virtual  institute  the  same  as  conferences.  This  is  not  available 
at  the  moment. 
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•  Subject  Matter  Expert  (SME)  Contact  Data 

A  list  of  SME  contact  data  with  areas  of  responsibility,  biographies  and  references  could  be 
provided  by  the  virtual  institute  for  specific  research  areas.  This  is  not  available  on  the  RTO  web 
yet.  Each  project  is  responsible  for  its  own  contact  data  but  there  is  no  possibility  to  publish  this 
data  or  to  link  it  to  specific  issues. 

•  Checklists  or  4 How  To’  Guides 

Checklists  or  how  to  guides  for  the  conduct  of  projects  or  part  of  it  (e.g.  experiment  set  up  or  data 
sharing  agreements)  could  be  made  available  by  the  virtual  institute.  This  is  only  partly  available 
from  RTO  at  the  moment. 

•  Legal  Guidance 

Information  regarding  legal  issues  like  intellectual  property  rights  or  security  should  be  published 
by  the  virtual  institute  for  different  countries  and  alliances.  A  reference  to  data  exchange 
agreements  and  procedures  should  be  established.  This  is  not  available  at  the  moment. 

•  Experiment  Data  Sharing 

The  raw  data  of  experiments  could  be  made  available  to  the  community  for  specific  analysis  or 
hypotheses  checking.  Such  functionality  is  momentary  not  available  and  has  to  be  established  by 
each  project  by  itself. 

•  Support  of  (Distributed)  Experiments 

The  virtual  institute  can  support  (distributed)  experiments  in  many  different  ways.  It  starts  with 
giving  advice  to  the  experiment  set  up  based  on  experience.  It  can  also  provide  tools  for  the 
execution  of  the  experiment.  This  can  be  a  virtual  environment  provided  by  the  virtual  institute  or 
simulations,  data,  scenarios,  etc.  The  virtual  institute  could  also  support  the  evaluation  and  analysis 
of  the  experiment. 

4.6  DATA  EXCHANGE 

Security  issues  and  intellectual  property  rights  have  an  impact  on  the  storage  and  exchange  of  data  in  a 
virtual  institute.  Participating  nations  and  organizations  have  to  agree  on  Memoranda  of  Understanding 
(MOU),  which  constitute  the  rules  for  the  operation  of  the  virtual  institute  (umbrella  MOU).  For  each 
specific  experiment  or  other  project  an  own  MOU  between  the  participating  parties  have  to  be  agreed  on. 
Here  it  would  be  helpful  if  the  virtual  institute  could  provide  an  overview  on  already  existing  data 
exchange  agreements,  which  could  be  used. 

The  virtual  institute  must  protect  intellectual  property  rights  (e.g.  commercial  simulation  systems  used  in 
experiments). 

Besides  the  legal  side  of  security  and  intellectual  property  rights  the  virtual  institute  must  provide 
technical  means  to  assure  that  data  is  stored  and  exchanged  in  the  right  way  (e.g.  encrypted  network).  This 
could  make  it  necessary  to  install  separate  infrastructure  for  different  projects.  Even  if  existing  networks 
like  the  Combined  Federated  Battle  Lab  (CFBL)  network  are  used,  additional  costs  might  apply.  Another 
issue  with  secured  networks  is  that  they  can  only  be  operated  in  a  secure  environment,  which  makes  the 
information  exchange  with  and  participation  of  independent  research  institutes  and  universities  difficult. 

4.7  CURRENT  RTO  WEBSITE 

The  current  RTO  website  serves  two  purposes:  it  represents  RTO  with  its  work  to  the  public  on  the 
Internet  and  it  provides  some  of  the  virtual  institute  functionality  to  the  RTO  with  its  Panels  and  technical 
teams.  Therefore  the  RTO  Website  is  structured  to  provide  access  to  information  and  services  for  the 
general  public  and  for  registered  users  after  login  (protected  information  up  to  NATO  Unclassified  and 
private  collaborative  workspaces). 
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The  home  page  provides  the  latest  RTO  news  and  navigation  to  the  various  other  parts  of  the  site. 
The  information  on  the  home  page  is  publicly  available  and  hence  no  logon  is  required.  The  General 
Documentation  page  provides  access  to  RTO  documentation  and  is  divided  into  Public  and  Protected 
areas.  For  example,  under  the  public  area  there  are  documents  like  the  RTO  Strategy,  RTO  Charter  and 
some  general  briefings  on  the  RTO.  Examples  of  protected  information  are  the  RTO  Operating 
Procedures,  RTO  Programme  of  Work,  the  RTO  Rolling  Plan  and  the  RTA  internal  telephone  directory. 

Full  details  of  all  past,  current  and  some  planned  RTO  activities  are  available  from  a  single  searchable 
activities  page.  By  selecting  a  particular  activity,  it  is  possible  to  see  also  other  related  areas  of  RTO 
research  through  a  taxonomy  based  linked  database  along  with  links  to  associated  publications  or  future 
meetings,  depending  upon  the  status  of  the  activity.  The  majority  of  information  on  RTO  Activities  is 
publicly  available,  although  upon  logon  users  will  have  access  to  more  information  about  each  activity, 
including  access  to  a  printable  version  of  the  Technical  Activity  Proposal  (TAP). 

A  full  calendar  of  forthcoming  symposia,  workshops  and  lecture  series  is  available  from  the  website, 
where  the  full  theme  in  both  English  and  French  is  available  along  with  the  ability  to  enrol  through  the 
RTO  on-line  enrolment  process.  In  addition,  prospective  Authors  can  submit  their  abstracts  via  the  RTO 
Website  for  review  by  the  respective  Programme  Committee. 

The  RTO  website  provides  for  full  download  of  all  RTO  and  AGARD  reports  up  to  NATO  Unclassified 
plus  an  unclassified  abstract  of  reports  of  a  higher  classification  (up  to  NATO  Secret).  The  website  also 
has  the  usual  full  search  and  retrieval  capability.  Access  to  the  full  text  of  publications  is  dependent  upon 
the  classification  of  each  report. 

There  is  a  page  dedicated  to  the  RTB  where  users  can  view  decision  sheets,  presentations  and  other  related 
information  for  both  past  and  future  Board  meetings.  This  information  is  protected  and  as  such  requires 
user  authentication. 

A  page  has  been  dedicated  to  information  about  the  RTO  National  Coordinators.  This  page  contains  a  list 
of  the  National  Coordinator  for  each  nation;  part  of  this  information  is  protected. 

Under  each  Panel/NMSG,  there  are  several  sub  areas  containing  information  as  follows: 

•  An  explanation  about  the  mission  and  scope  of  each  Panel  plus  latest  news. 

•  A  full  list  of  activities  for  each  Panel  is  listed,  along  with  details  concerning  the  activity. 

•  A  full  list  of  forthcoming  events  for  each  Panel  is  listed,  along  with  the  ability  to  enrol  on-line. 

•  A  full  list  of  the  published  Scientific  Publications  from  each  Panel  is  listed,  and  provides  for  full 
download  of  all  reports  up  to  NATO  Unclassified  plus  an  unclassified  abstract  of  reports  of  a 
higher  classification  (up  to  NATO  Secret).  These  pages  are  essentially  the  same  as  the  main  RTO 
Publications  page  but  filtered  to  include  only  those  publications  for  the  associated  Panel. 

•  The  structure  of  each  Panel  is  explained  along  with  a  list  of  the  national  Panel  members.  This 
information  is  protected  and  hence  requires  user  authentication. 

•  Pages  for  the  download  of  relevant  Panel  documents,  for  example  the  Decision  Sheets  of  Panel 
Business  Meetings.  This  information  is  protected  and  as  such  requires  user  authentication. 

The  NATO  Science,  Technology  and  Research  Network  (STARNET)  is  build  to  facilitate  access  to 
information  elements  existing  worldwide,  in  terms  of  science,  technology  and  overall  research;  it  is  a 
database  of  Web-based  data  sources,  which  will  allow  comprehensive  and  sophisticated  searches. 
STARNET  is  designed  as  a  virtual  library  to  provide  a  “one  stop”  information  resource  for  policy  makers, 
program  managers,  scientists,  engineers  and  researchers.  It  has  been  designed,  as  a  system  that  can  be 
adapted  to  address  specific  information  needs  as  they  arise  within  the  NATO  community. 
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STARNET  is  composed  of  seven  scientific  areas  or  nodes  as  listed  on  the  left  of  the  page.  Each  node  has 
specific  topics  assigned,  with  its  own  inventory  of  web-available  information  resources. 

These  resources  are  chosen  based  on  their  relevancy  to  the  specific  topics  assigned  to  each  of  the  nodes. 

In  addition  to  the  open  and  protected  information,  the  Website  provides  a  series  of  closed  private 
collaborative  workspaces  for  specific  members  of  the  RTO  Community.  The  secure  closed  workspaces 
correspond  largely  to  the  structure  of  the  RTO,  being  mainly  for  Level  3  Task  Group  activities. 

Each  workspace  has  the  following  functionality: 

•  Workspace  News  -  general  news  area  for  each  workspace; 

•  File  Explorer  -  Shared  Files  and  Folders; 

•  Threaded  Discussion  -  Threaded  Message  Discussion  Board; 

•  List  of  Contacts  -  Specific  to  each  Workspace; 

•  Useful  Web  Resources  -  Area  for  building  a  library  of  links  to  useful  web  based  resources  specific 
to  each  workspace;  and 

•  Related  Research  by  Taxonomy  -  Link  to  the  Taxonomy  based  research  matching  algorithm  to 
view  possible  areas  of  related  research  within  the  NATO  R&T  Community  (currently  limited  to 
RTO  and  ACT  LTCRs). 

The  workspace  environment  uses  the  Secure  Socket  Layer  (SSL)  version  3.0  protocol  and  is  firewall 
protected  using  architecture  accredited  by  the  NATO  Office  of  Security  and  as  such  can  be  accessed  from 
computer  connected  to  the  Internet  with  access  to  HTTP,  and  SSL  (v3)  protocol. 


4.8  VI  APPLICATIONS 

The  need  for  a  multi-national  human  behavior  research  virtual  institute  is  apparent  in  many  areas,  most 
distinctly  in  the  area  of  multi-national  command  and  control  (C2).  Within  this  environment,  military 
individuals  from  various  nations  must  work  in  a  complicated  coalition  to  assemble  effective  teams.  Within 
these  teams,  there  are  several  aspects  that  play  significant  importance  in  the  formation  of  effective  teams. 
In  addition,  the  stress  of  the  complex  C2  environment  in  which  the  multi-national  team  is  employed  puts  a 
greater  demand  on  the  team  to  work  as  efficiently  and  effectively  as  possible.  To  enable  multiple  nations 
to  form  the  most  effective  teams  as  quickly  as  possible  in  a  joint  command  team  C2  environment,  testing 
is  required  to  show  the  strengths  and  weaknesses  of  joint  team  ability.  Using  these  strengths  and 
weaknesses,  team  members  can  collaborate  and  communicate  in  a  more  effective  manner  and  the  joint 
command  team  will  perform  more  efficiently. 

4.8.1  Exemplar  Uses  of  the  Virtual  Institute 

A  wide  variety  of  experiments  can  be  conducted  at  modest  cost  and  with  scientific  worth,  which  in  turn 
can  result  in  better  operational  performance.  In  this  section  we  will  discuss  a  few  of  the  types  of 
experiments  that  can  be  executed  within  the  VI.  One  type  of  experiment  that  can  be  performed  is  a 
background  experiment  to  determine  the  differences  between  the  NATO  nations  as  regards  their  military 
and  national  cultures  and  encode  these  differences  in  a  knowledge  base.  One  key  aspect  of  the  differences 
between  cultures  is  the  socialization  process  and  the  differences  between  customs  and  attitudes. 
To  accomplish  socialization  is  important  but  is  also  very  difficult.  While  NATO  nations  all  work  together 
to  cooperate,  true  operational  effectiveness  cannot  be  achieved  until  socialization  among  them  is  achieved, 
which  means  that  culture  differences  must  be  overcome  and  also  misunderstandings  due  to  word  usage 
and  custom  that  are  unfamiliar  to  other  parties  must  be  avoided.  Traditionally,  socialization  occurs  as  a 
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result  of  many  conversations  and  interactions  that,  in  bulk,  permit  some  portion  of  the  information 
necessary  for  the  development  of  mutual  trust  and  interoperability  to  be  exchanged  but  while  it  is  not  clear 
how  long  the  normal  process  takes,  the  time  duration  is  certainly  on  the  order  of  days  or  weeks.  Part  of  the 
reason  for  the  length  of  time  required  for  socialization  is  that  it  is  a  hit  and  miss  process  and  is  usually 
accomplished  over  lengthy  periods  of  time  among  people  with  similar  cultures.  The  challenge,  then,  is  to 
conduct  experiments  to  determine  how  to  socialize  people  rapidly  across  multiple  cultures  when  at  this 
time  it  is  not  clear  which  knowledge  about  other  cultures  is  most  crucial  to  an  operation  or  to  an  effective 
coalition  action.  The  VI  can  be  used  to  conduct  binary  and  n-ary  experiments  to  determine  effective  means 
for  accelerating  the  socialization  process  and  making  socialization  actions  effective.  The  VI  can  be  used  to 
design  and  execute  experiments  that  measure  development  of  shared  comprehension  of  situation  across 
coalition  and  cultures  and  determine  what  portions  of  network-centric  benefits  only  accrue  when  culture 
and  national  differences  are  considered.  For  example,  in  support  of  the  twin  issues  of  trust  establishment 
and  coalition  member  socialization,  the  virtual  institute  can  be  used  to  help  coalition  members  to  rapidly 
arrive  at  a  common  understanding  of  the  coalition  members’  and  coalition’s  information  security 
capabilities,  information  security  operations  capabilities,  and  NCW  capabilities.  As  the  coalition  partners 
increase  their  degree  of  operational  effectiveness,  their  reliance  upon  network  resources  to  retain  this  high 
degree  of  operational  effectiveness  increases.  However,  this  degree  of  coalition  operational  effectiveness 
can  be  degraded  if  the  network  is  attacked  and,  subsequently,  confidence  in  NCW  resources  and 
capabilities  starts  to  decrease.  The  degradation  can  pose  a  problem  for  coalition  NCW  operations.  We  can 
use  the  virtual  institute  to  determine  how  different  cultures  respond  to  this  circumstance  and,  thereby, 
prepare  procedures  for  it  and  develop  doctrine  to  allow  coalition  partners  to  deal  with  this  eventuality. 

Even  if  traditional  socialization  cannot  be  replaced,  the  VI  can  provide  a  continuously  at-hand  capability 
for  interactions  among  the  nations,  which  can  form  the  basis  for  understanding  and  accelerate 
socialization.  Wide  area  simulation  among  coalition  partners  on  a  regular  basis,  for  example  monthly, 
would  speed  up  the  socialization  process  but  is  only  one  step.  The  VI  can  also  be  used  to  determine  the 
cultural  impact  on  decision  making  by  answering  questions  like  the  following: 

1)  Do  different  cultures/militaries  use  different  information  to  make  decisions  when  faced  with  same 
situation  and/or  do  they  use  the  same  information  differently? 

2)  How  should  information  be  shared  in  light  of  the  differences  in  culture  and  custom  so  that  all 
partners  perceive  the  situation  the  same  way  and  reach  same  conclusion. 

The  VI  can  be  used  to  investigate  and  research  key  technical  and  cultural  issues  that  must  be  addressed  if 
NCW  is  to  be  successful  for  the  US  and  coalition  members  in  a  coalition  environment.  Specifically,  issues 
related  to  the  type  of  communication  that  is  needed,  bandwidth,  data  security,  redundancy,  and 
communication  flexibility  can  be  determined  using  the  VI.  Experiments  in  the  VI  to  determine  how  to 
maintain  cohesion  and  the  will  to  fight  across  cultures  in  a  distributed  command  can  also  be  conducted  as 
well  as  experiments  to  determine  the  new  leadership  challenges  that  emerge  in  NCW  that  must  be 
overcome.  And,  since  the  coalition  will  be  comprised  of  many  different  types  of  computer  and 
communication  security  mechanisms,  the  VI  can  be  used  to  conduct  experiments  to  determine  how  to 
securely  exchange  all  relevant  and  supportive  data  across  a  coalition  while  still  preserving  each  nation’s 
NCW  capabilities  and  a  secure  information  technology  infrastructure.  Additionally,  the  VI  can  be  used  to 
assess  and  model  the  performance  of  individuals  and  of  the  team  (groups,  command  post,  etc.)  when  the 
information  flow  is  under  cyberattack  and  the  information  elements  that  are  used  by  individuals  and  teams 
cannot  be  trusted.  The  VI  can  also  be  used  to  determine  and  model  how  to  counteract  the  effects  of 
uncertain  information  and  to  assess  which  information  is  of  the  most  crucial  importance  under  a  variety  of 
situations.  These  models,  and  the  techniques  developed  to  counteract  uncertainties  that  arise  due  to 
cyberattack,  can  have  great  utility  for  coalition  operations  and  for  operations  that  have  short  execution 
timelines.  To  our  knowledge,  the  VI  would  be  the  first  and  only  environment  in  which  the  required 
experiments  and  exercises  could  be  conducted  securely  using  wide  varieties  of  coalition  members  and 
individuals  that  are  performing  individual  and  team  tasks. 
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The  security  of  the  information  technology  infrastructure  points  to  the  need  to  investigate  information 
assurance  and  software  protection  as  they  relate  to  human  behavior,  socialization,  team  building,  cohesion, 
and  decision-making.  Until  recently,  the  need  for  information  assurance  and  application  software 
protection  and  security  for  its  data  has  been  addressed  through  efforts  to  provide  security  for  the 
simulation  environment  using  network  level  resources  or  operating  system  level  resources.  However, 
the  intended  targets  of  the  attacks,  the  application  software  and  its  data,  have  no  inherent  protection  and 
rely  solely  upon  network-based  and  operating  system  based  defences  for  their  protection.  Even  with  the 
use  of  strong  encryption  technology,  these  two  forms  of  cyber-defence  have  failed  to  secure  applications 
from  a  variety  of  attacks.  The  record  of  successful  attacks  demonstrates  that  simply  placing  security 
defence  technology  in  the  network  and  the  operating  system,  while  necessary,  is  not  sufficient  to  provide 
protection  and  security  for  the  application  software  and  data  involved  in  distributed  simulation 
environments.  It  is  increasingly  apparent  that  software  and  data  must  be  protected  using  a  variety  of 
technologies  that  could  be  embedded  in  the  application  software  and  data. 

Application  security,  which  is  the  security  embedded  in  application  software  and  data,  will  become  a  more 
important  part  of  modeling  and  simulation  environments,  and  hence  the  VI,  as  the  environments  increase 
in  their  fidelity  and  as  more  locations  participate  in  simulation  exercises  and  experiments.  The  increase  in 
fidelity  will  increase  the  value  of  both  the  simulation  software  and  of  the  data  and  will  make  them 
increasingly  valuable  targets  for  attack.  The  increase  in  value  will  also  result  in  more  resources  being 
devoted  to  obtaining  the  information  contained  in  the  simulation  software  and  data.  Encrypting  the  data 
while  in  transit  between  hosts  is  not  sufficient  to  address  the  spectrum  of  security  threats  to  be  countered. 
Threats  to  the  data  exist  while  it  resides  on  a  host  in  unencrypted  form  and  also  to  the  software  that 
comprises  the  simulation  environment.  These  threats  arise  because  the  network  and  operating  system 
portions  of  the  simulation  environment  do  not  provide  security  to  insure  the  protection  of  the  data  and 
software.  Therefore,  defences  for  applications  and  their  data  are  needed,  and  these  capabilities  should 
naturally  be  placed  in  the  application  because  the  application  layer  provides  a  logical  location  for  the 
placement  of  additional  protection  capabilities.  Note  that  there  are  pressing  research  needs  that  hinder 
widespread  adoption  of  application  security  technologies.  Some  of  these  research  needs  include  the 
development  of  techniques  that  allow  an  application  to  detect  attempted  compromise,  techniques  to  allow 
an  application  to  detect  tampering,  and  metrics  that  permit  the  objective  measurement  and  comparison  of 
software  and  data  protection  techniques. 

In  our  view,  there  are  two  main  technical  objectives  for  practical  software  protection  as  they  relate  to  the  VI: 

1)  To  make  the  task  of  violating  an  application  so  difficult  that  the  effort  needed  to  compromise  an 
application  is  too  high  for  most  attackers;  and 

2)  To  make  the  task  of  breaking  into  an  application  so  time-consuming  that  most  attackers  who  could 
accomplish  a  penetration  give  up. 

As  a  complementary  requirement,  those  attacks  that  do  succeed  must  be  quickly  detected  in  order  to  limit 
the  damage  inflicted.  To  attain  these  objectives,  protected  software  should  exhibit  five  critical  properties: 

1)  It  should  be  difficult  to  change  the  overall  functionality  of  the  software  via  small,  local  changes  to 
the  code; 

2)  It  should  be  difficult  to  analyze  the  software’s  control  and  data  flows; 

3)  It  should  be  difficult  to  determine  the  software’s  runtime  behaviour; 

4)  It  should  be  difficult  to  attack/subvert  the  software  via  its  execution  environment/host  platform;  and 

5)  The  protection  should  employ  strong  cryptographic  technologies. 

Furthermore,  defending/protecting  software  against  attacks  and  compromise  require  that  applications  be 
aware  of  their  environment  and  must  cautiously  interact  with  that  environment.  Inputs  from  users,  whether 
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those  users  are  human  or  other  software  applications  must  be  carefully  scrutinized.  An  important  adjunct 
to  these  principles  is  security  threat  modeling,  especially  as  it  relates  to  human  behavior  and  decision¬ 
making.  The  Unified  Modeling  Language  (UML)  can  be  used  to  portray  and  modeling  application  security 
threats  in  a  manner  that  is  clear  and  standard  as  well  as  extensible.  An  accurate  model  of  the  threats  that  an 
application  will  face  and  must  defeat  is  an  essential  first  step  in  determining  the  defences  needed  by  an 
application. 

Because  of  the  importance  of  coalition  operations,  near-term  research  must  also  address  the  problem  of 
protection  of  software  across  the  entire  spectrum  of  hardware  platforms,  software,  automated  insertion  of 
protection  techniques  into  software  middleware,  and  integration  of  application  security  techniques  with 
operating  system  and  network-based  defensive  capabilities  is  important.  For  the  modeling  and  simulation 
community,  the  protection  of  VI  software  is  also  important  and  poses  several  challenges  in  terms  of 
performance  impact,  stealthiness  of  protection,  and  the  proper  mix  of  protection  techniques  is  important 
for  the  VI,  especially  as  they  relate  to  the  timeliness  of  decision-making,  human  behaviors,  cultural 
differences,  team  building,  and  socialization. 

One  of  the  clearest  and  most  pressing  priorities  is  the  need  for  a  lexicon  and  ontology  of  modeling  and 
simulation  terms  as  they  relate  to  the  VI.  These  test  suites  need  to  include  benchmarks,  metrics, 
an  evaluation  system  (or  rating  system),  and  security  testing  scenarios  that  would  allow  the  modeling  and 
simulation  community  to  evaluate  the  efficacy  of  various  software  protection  techniques  in  modeling  and 
simulation  applications  and  environments  and  for  the  way  they  affect  human  behavior.  In  addition  to  the 
standardization  needs,  there  is  also  the  need  for  information  that  can  guide  the  modeling  and  simulation 
community  in  combining  network,  operating  system,  and  application  security  capabilities  in  a  mutually 
reinforcing  manner  so  as  to  address  threats  in  a  way  that  have  minimal  impact  upon  performance. 
This  need  is  especially  important  given  the  modeling  and  simulation  communities,  and  the  Vis,  reliance 
upon  networks  and  the  sensitive  data  transmitted  across  them.  As  a  result,  there  is  a  need  to  protect 
data  assets  in  a  distributed  simulation  environment.  Because  a  distributed  simulation  operates  within  a 
continuously  varying  environment,  the  modeling  and  simulation  community,  and  the  VI,  would  benefit  from 
software  protection  techniques  that  can  be  composed  during  execution  in  concert  with  network  and  operating 
system  security  techniques  to  maintain  a  desired  degree  of  security  for  a  distributed  simulation  even  in  the 
face  of  attack.  To  achieve  this  goal,  even  in  part,  standardization  work  can  be  performed  in  the  VI  that  can  be 
used  to  enable  components  of  a  distributed  simulation  environment  to  establish  trust  among  themselves, 
exchange  security  interface  information  and  security  requirements  information,  and  continuously  monitor  the 
application  security  performance  of  the  other  components  of  their  application  and  of  the  distributed 
simulation  as  a  whole  is  necessary. 

A  challenge  related  to  NCW  that  can  be  addressed  using  the  virtual  institute  are  the  challenges  posed  by 
Effects  Based  Operations  (EBO)  and  the  EBO  deployment  and  implementation  considerations  and  tactics 
that  should  be  used  by  a  coalition  against  multi-cultural  enemies.  Because  EBO  is  directed  toward  the 
enemy  and  their  perceptions  of  the  battlespace  (and  thereby  indirectly  their  will  and  capability)  while  not 
distorting  coalition  members’  perceptions  and  understanding,  it  is  important  to  understand  how  to  both 
affect  perceptions  of  the  enemy  and  avoid  affecting  perceptions  of  coalition  partners.  The  virtual  institute 
can  be  used  to  address  both  of  these  challenges  regarding  understanding  of  human  perception  by 
determining  how  to  conduct  EBO  in  a  manner  that  reinforces  coalition  will  and  capability  while  also 
undermining  those  of  the  enemy.  The  VI  can  be  used  to  understand  and  build  models  of  the  human 
behavior  of  the  enemy,  determine  how  that  culture  operates,  its  values,  and  how  an  understanding  of  the 
culture  can  be  used  in  EBO. 

All  of  these  information  security  measures  should,  of  course,  be  aimed  at  determining  the  types  of 
technical  solutions  that  are  needed  to  shorten  the  decision  cycle  across  and  within  each  component  of  the 
coalition.  Shortening  the  decision  cycle  requires  development  of  a  common  understanding  of  plans  and 
procedures,  development  of  a  common  mental  model  of  the  battlespace,  as  well  as  means  for  raising  and 
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maintaining  confidence  and  morale  of  coalition  partners  in  commanders  and  coalition  technologies,  all  of 
which  are  experiments  that  can  be  conducted  at  reasonable  cost  in  the  VI.  The  VI  can  be  used  to  host 
experiments  that  assess  different  means  for  maintaining  confidence  in  information  security  measures 
within  the  coalition. 

Given  the  disparities  in  information  technologies  and  standards  used  for  communication  by  the  potential 
coalition  members,  achieving  information  synchronization  and  interoperability  in  spite  of  differences  in 
information  velocity  and  transport  between  coalition  members.  The  differences  arise  and  are  a  challenge 
because  each  force  has  its  own  natural  operational  tempo  and  own  natural  information  velocity  within  its 
own  national  information  technology  infrastructures.  Clearly,  these  characteristics  will  not  match  across 
cultures  and  so  devising  means  that  allow  coalition  members  to  adapt  to  the  differences  in  a  manner  that 
does  force  everyone  to  operate  at  the  slowest  member’s  pace.  Furthermore,  qt  this  time,  multi-national 
simulation  experience  indicates  that  different  cultures/militaries  use  different  information  to  make 
decisions  when  faced  with  the  same  situation.  However,  each  nation  and  culture  purports  to  make  use  of 
identical  information  in  its  decision  making  process.  Addressing  this  seeming  contradiction  will  allow  us 
to  determine  the  priority  for  the  information  that  has  to  flow  into  each  coalition  partner  from  all  other 
partners.  Proper  prioritization  will  insure  that  each  partner  has  the  right  information  that  the  partner  needs 
at  the  right  time  in  order  for  it  to  act  upon  the  information  at  hand  in  a  way  that  is  consistent  with  their 
culture  and  decision-making  style.  The  VI  is  can  host  the  experiments  required  to  determine  how  to 
address  these  problems  in  a  manner  that  minimizes  impact  upon  the  decision-making  cycle.  The  virtual 
institute  provides  a  capability  for  assessing  the  effectiveness  of  data  transport  between  coalition  partners 
and  a  testbed  for  evaluating  new  solutions  as  they  are  developed.  The  data  transport  capability  can  be  met 
using  a  set  of  XML-based  battle  management  languages  (BML)  for  data  transport  because  an  XML-based 
approach  helps  to  build  a  capability  for  translating  between  the  BMLs.  There  would  be  one  BML  for  the 
US  forces,  one  for  each  coalition  member,  and  one  common  form  that  serves  as  an  intermediate  form  to 
handle  translation  from  one  BML  to  another.  Using  this  approach  minimizes  the  amount  of  work  required 
when  a  new  coalition  member  is  added  or  when  new  aspects  need  to  be  added  to  the  data  transmission 
capability.  An  XML-based  approach  will  allow  the  languages  to  capture  the  nuances  of  the  different 
cultures  as  well  as  nuances  in  tactics  and  operational  concepts.  The  battle  management  language  would 
need  to  have  the  following  capabilities: 

1)  Is  secure; 

2)  Supports  information  sharing; 

3)  Is  secure  at  multi-levels  across  coalition  members; 

4)  Allows  culture  and  nuance  to  be  explicitly  captured  and  refined  in  the  coalition  battle  management 
language; 

5)  Can  express  any  directive,  operations  order,  etc.,  including  peacekeeping  operations; 

6)  Supports  change  to  the  battle  management  language  including  the  ontology  and  taxonomy  and 
element  tags; 

7)  Allows  multiple-way  mapping  between  coalition  member  concepts;  and 

8)  Supports  web  services  but  also  can  function  with  more  traditional  approaches. 

The  need  to  move  data  among  coalition  partners  also  raises  some  technical  and  policy  issues  that  the  VI 
can  be  used  to  address.  The  following  issues  as  being  the  most  crucial  at  this  time  and  can  be  investigated 
using  the  virtual  institute: 

1)  Determining  what  data  needs  to  move; 

2)  Determining  how  to  accommodate  bandwidth  mismatches  between  partners; 
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3)  Determining  how  to  accommodate  command  structure  mismatches  and  how  to  route  information 
to  the  desired  recipient  when  this  occurs; 

4)  Determining  how  to  automatically  establish  security  boundaries  and  provide  coalition  partners 
with  available  services  and  credentials;  and 

5)  Determining  how  to  show  a  need  to  know  and  need  to  share  data  and  services  with  coalition 
partners. 

The  virtual  institute  can  provide  an  environment  for  testing  the  technologies  and  policies  required  to 
address  these  challenges  and  can  provide  an  environment  where  solutions  can  be  demonstrated  under 
realistic  conditions. 

The  virtual  institute  can  also  be  used  to  develop  human  behavioral  models  of  coalition  cohesion  and  models 
of  enemy  effects  based  operations.  Given  the  rate  of  change  in  technologies  and  the  acceleration  in  the  rate  at 
which  coalition  cultures  must  socialize  and  interact,  the  virtual  institute  provides  a  unique  venue  for 
conducting  experiments  to  identify  future  and  emerging  technologies  that  could  be  exploited  to  rapidly 
identify  salient  background  information  about  individual  coalition  members  as  well  as  other  units  in  the 
coalition  during  the  run-up  to  joint  operations.  The  experiments  should  concentrate  on  addressing  how  the 
new  and  novel  aspects  of  network-centric  warfare  that  cause  interaction  challenges  and  command  and 
control  challenges  (such  as  different  technologies,  cultures,  notions  concerning  the  relative  importance 
information,  reaction  to  decrease  in  NCW  capabilities,  reaction  to  reduction  in  network  access,  and  means 
used  to  deal  with  high  taskload)  can  be  remediated  using  new  technologies  or  new  approaches  to  interaction 
and  command  and  control.  The  experiments  can  be  used  to  determine  the  salient  things  at  each  level  of 
command  and  across  a  coalition  and  determine  how  technologies  can  be  used  to  the  greatest  advantage  and 
which  technologies  and  techniques  provide  the  least  benefit  or  are  counterproductive.  The  experiments  can 
be  used  to  determine  how  the  technologies  and  techniques  need  to  be  changed  to  accommodate  different 
types  of  operations  and  coalitions.  Example  experiments  include,  but  are  not  limited  to  the  following. 
Do  personal  information  sharing/exchange  technologies  like  Facebook  and  MySpace  help  or  hinder  the 
development  of  cultural  understanding  and  mutual  trust.  Do  video-teleconferences  actually  supplant  face-to- 
face  meetings,  reinforce  them,  or  are  personal  encounters  more  important  than  ever  in  order  to  conduct  fast- 
paced  actions?  What  value,  if  any,  do  text-messaging  and  video-messaging  have  in  building  a  sense  of  trust, 
community,  and  jointness?  Which  technologies  and  techniques  best  counter  the  expected  enemy’s 
operations,  especially  effects-based  operations  (EBO),  against  coalition  forces?  Which  EBO  techniques 
would  be  of  the  most  advantage  to  the  enemy  in  developing  discord  among  the  coalition?  What  techniques 
should  be  best  employed  to  maintain  coalition  cohesion  and  unity  of  effort?  Conversely,  the  techniques  that 
are  counterproductive  must  be  identified. 

4.8.2  Assessing  Decision  Support 

The  virtual  institute  provides  an  experimental  environment  for  measuring  the  effectiveness  of  data 
transport  and  the  needs  outlined  in  the  preceding  paragraphs.  The  experiments  will  be  able  to  be  used  to 
determine  the  technologies  and  policies  that  provide  the  greatest  benefit  to  users,  at  least  in  terms  of 
timeliness  of  arrival  of  data,  which  is  especially  important  within  the  context  of  network  centric  warfare. 
Clearly,  the  ability  of  a  network-centric  force  to  operate  effectively  depends  upon  its  ability  to  move  data 
through  the  force  effectively  and  efficiently.  However,  the  obvious  accuracy  of  this  statement  does  not 
provide  sufficient  guidance  to  effectively  assess  the  quality  of  the  data  flow,  to  develop  and  assess 
doctrine  and  policy,  and  to  identify  areas  within  the  organization  where  changes  in  policy  and  data  flow 
processes  can  be  changed  to  the  greatest  positive  effect.  To  help  provide  the  necessary  guidance, 
we  developed  a  foundation  for  a  formal  means  for  assessing  data  movement  within  an  organization. 

Within  a  network  centric  organization,  there  are  two  sets  of  entities:  sources  of  data  and  recipients  of  data. 
Within  the  organization,  let  r  be  the  set  of  data  recipient  entities  and  allow  them  to  be  arbitrarily  and 
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uniquely  labeled  from  1  to  n.  Within  the  same  organization,  let  s  be  the  set  of  data  source  entities  and 
allow  them  to  be  arbitrarily  and  uniquely  labeled  from  1  to  m.  Let  Ir  be  the  data  required  by/destined  for  a 
particular  recipient  of  data  and  n  be  the  number  of  data  recipients  and  let  Is  be  the  data  sent  from  any 
source  of  data  and  m  be  the  number  of  data  sources.  Then,  Ir  <—  Is  defines  the  instantaneous  data  volume 
(in  bytes)  between  all  sources,  s,  and  recipients,  r,  of  data.  We  can  then  define  Ii  as: 


<r-  r  7 


(1) 


Ii  is  the  total  instantaneous  volume  of  data  moving  from  all  sources  of  data  to  all  recipients  of  data  within 
an  organization  at  any  given  time.  This  definition  is  source-recipient  topology  neutral,  technology  neutral, 
and  bandwidth  independent.  Based  this  definition  for  Ii,  it  is  clear  that  for  a  network-centric  force  to  be 
effective,  its  data  capacity  must  accommodate  peak  demands  for  transmission  of  operational  data  in 
conjunction  with  simultaneous  peak  demand  for  transmission  of  network  management  data.  Using  Ii, 
we  define  the  instantaneous  data  velocity  xn  within  an  organization  at  a  given  time  x  in  Equation  2: 

.x  =  (llT-llT.l)/llx.1  (2) 


The  mean  data  velocity  for  an  organization  over  a  given  time  interval  y  is  defined  as  shown  in 
Equation  3.  A  network  centric  organization  should  strive  to  attain  a  high  value  for  wm  as  well  as  a  high 
value  for  w  during  important  periods  of  activity.  Two  other  types  of  important  information  flows  are  the 
ability  of  the  organization’s  infrastructure  to  give  expedited  handling  to  important  (or  priority)  information 
and  the  ability  of  an  organization’s  infrastructure  to  provide  communication  connectivity  between  high 
priority  communicants.  To  determine  if  the  organization’s  infrastructure  addresses  these  two  types  of 
information  handling  appropriately,  we  first  determine  how  well,  on  average,  the  organization’s 
infrastructure  supports  information  flows  between  all  senders  and  recipients  of  information.  Let  the 
expression  <—  Si  ^  0)  be  used  to  indicate  that  there  is  message  traffic  between  a  given  recipient  and 
transmitter  of  information.  Let  I2  be  the  weighted  data  movement  time  required  for  data  to  move  from  all 

sources  to  all  recipients  within  an  organization  for  a  given  time  period  x,  then  I2  is  defined  in  Equation  4: 
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Clearly,  for  a  network-centric  organization  to  be  effective,  I2  must  be  as  large  as  possible  for  all  time 

periods  and  I2  must  also  be  maximized  over  all  time  intervals.  I2  for  a  given  number  of  time  intervals,  y, 
is  calculated  as  shown  in  Equation  5: 
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To  insure  that  I2^  and  I2  are  maximized,  there  must  be  no  contention  for  bandwidth  within  the  organization 

and  operational  and  network  management  data  must  be  transmitted  with  equal  promptness.  While  efficient 
transmission  of  data  is  crucial  to  network  centric  warfare,  not  all  data  is  of  equal  importance,  as  we  have 
indicated.  So,  even  though  I2  may  be  acceptable,  the  organization  may  not  be  operating  effectively  from  a 
network-centric  warfare  perspective  because  higher  priority  information  is  not  given  corresponding  higher 
precedence  for  transmission  through  the  organization’s  infrastructure.  Therefore,  we  require  a  metric  to 
assess  how  well  the  organization  manages  the  transmission  of  priority  data;  the  overall  efficiency  of  data 
transmission  for  priority  information  is  indicated  by  the  value  of  I3.  Let  I3p  be  the  average  time  for  priority 

data  to  move  from  all  sources  to  all  recipients  of  data  of  that  given  priority  within  any  given  time  period  x 
within  an  organization.  Assume  that  data  has  only  one  priority  or  that  it  is  handled  in  accordance  with  the 
highest  priority  level  assigned  to  it.  Assume  there  are  x  sets  of  priority  data  within  that  same  time  period, 
and  let  p  represent  a  given  data  priority  for  data  within  an  organization,  clearly  p  can  assume  any  value  in 
the  range  1  to  x.  Then  I3p  for  a  given  message  priority  p  at  a  time  x  is  defined  as  shown  in  Equation  6. 

I3  for  all  message  priorities  at  a  given  time  x  is  defined  in  Equation  7. 
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Let  I4f  be  the  time  differential  between  the  time,  tn,  when  data  (of  all  priorities)  is  needed  by  a  recipient 
and  the  time,  ta,  when  it  is  received  by  the  recipient  in  a  given  time  period.  Then  I4f  for  a  given  recipient  r 
during  a  given  time  period  is  defined  by  Equation  8: 
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and  I4  for  an  organization  with  n  recipients  within  that  same  time  period  is  defined  as  shown  in  Equation 
9: 
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Table  4-2 
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the  major  metrics/variables  defined  in  this  section  and  a  short  specification  for 
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Table  4-2:  Major  Metrics/Variables  and  Their  Definitions 


Metric/Variable 

Definition 

Ii 

The  volume  of  data  moving  from  all  sources  of  data  to  all  recipients  of 
data  within  an  organization  at  any  given  time. 

l2 

The  average  time  for  data  to  move  from  all  sources  to  all  recipients 
within  a  time  period. 

b 

The  average  elapsed  time  for  priority  data  of  a  given  priority  to  move 
from  all  sources  to  all  recipients  of  data  of  that  priority  at  any  given 
time. 

l4 

The  time  differential  between  the  time  when  data  is  needed  by  a 
recipient  and  when  it  is  received. 

U 

The  time  differential  between  the  time  when  data  is  needed  by  a 
recipient  and  when  it  received  by  the  data  recipient  for  a  given  time 
period  for  data  of  a  given  priority. 

U 

The  ability  of  the  infrastructure’s  performance  to  scale  when  data 
volume  increases. 

l7 

The  ability  of  an  organization’s  infrastructure  to  transport  information 
between  important  senders  and  important  recipients  of  information. 

Data  velocity  within  an  organization  at  a  time  x. 

V 

The  efficiency  of  the  movement  of  data. 

A  network-centric  force  has  the  capability  to  share  and  exchange  data  among  the  distributed  elements  of  the 
force  and  has  access  to  data  whenever  and  wherever  it  is  needed.  However,  without  metrics  we  are  unable  to 
readily  determine  when  or  if  the  organization’s  infrastructure  has  the  capability  to  support  network-centric 
operations.  The  above  measures  for  data  movement  within  a  network-centric  organization;  measures  that 
allow  us  to  assess  the  efficiency  of  data  flow,  its  velocity,  the  total  data  flow  at  any  time,  and  measures  to 
assess  the  ability  of  a  data  transport  system  to  respond  to  the  demands  placed  upon  it  during  the  course  of  an 
operation  when  it  is  under  cyber  attack  and  when  it  is  not.  One  clearly  important  parameter  within  a  network 
centric  organization  is  Ii.  The  data  transportation  system  must  be  able  to  satisfy  any  demands  placed  upon 
it  for  data  and  all  other  things  being  equal,  a  system  with  a  larger  Ii  is  better  than  one  with  a  smaller  Ii. 
Additionally,  the  data  transportation  system  should  strive  to  achieve  a  small  value  for  wT  under  all 

circumstances,  because  this  will  insure  that  the  system  can  handle  the  data  load  and  that  the  system  can  also 
respond  to  changes  in  demand  rapidly. 

4.8.3  Exemplar  Multi-Cultural  Experimentation  Using  the  VI 

The  multi-national  research  experiment  that  demonstrated  the  prototype  VI/HBR  can  be  used  to  investigate 
the  impact  of  culture  on  teamwork  in  both  national  and  multi-national  venues.  The  experimentation  would 
involve  conceptual  model  development  encompassing  the  impact  of  culture  on  joint  military  operations, 
extending  experimentation  methods  to  include  process  and  outcome  measures,  and  identifying  products  for 
national  use  to  improve  leader  and  team  performance  in  multi-cultural  environments.  This  experimentation 
can  be  executed  using  a  research  testbed  built  around  a  commercial  off  the  shelf  (COTS)  computer  game, 
such  as  Neverwinter  Nights™.  This  game-based  testbed,  funded  by  the  United  States  Department  of  Defense 
Modeling  and  Simulation  Office  (DMSO),  is  designed  to  satisfy  the  need  for  an  inexpensive,  standardized, 
research  instrument  able  to  explore  basic  research  questions  on  teamwork  skills,  situation  awareness, 
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decision  making,  task  effectiveness,  adaptability,  and  the  impacts  of  personality  and  cultural  traits  on  these 
issues.  A  single  experimental  design  can  be  executed  in  all  the  participating  nations  to  build  a  baseline  of 
responses  from  homogenous  military  teams.  Then,  that  design  can  be  executed  with  heterogeneous  teams 
comprised  of  military  officers  from  various  nations  participating  in  the  baseline  experimentation. 

The  HBR  Virtual  Institute  experiment  team  collaborated  with  two  NATO  RTO  groups  to  further  its 
experimental  planning  in  this  area:  RTG  SAS-050  and  an  HFM  Exploratory  Team.  The  VI/HBR  experiment 
team  utilized  the  main  research  product  of  SAS-050,  which  was  a  conceptual  model  of  Command  and 
Control  (C2).  The  NATO  VI/HBR  experiment  team  analyzed  and  applied  this  conceptual  model  and 
taxonomy  of  C2  variables  as  a  starting  point  for  the  variables  within  the  cultural  modeling  testbed  to  measure 
for  their  effects  on  teamwork  and  cultural  influences  to  teamwork  and  leadership.  In  cooperation  with  an 
HFM  Exploratory  Team,  SAS-053  collaborated  on  the  experimental  approach  to  test  leader  and  team 
adaptability  in  multi-national  coalitions  for  baseline  testing,  measurement,  and  analysis.  This  experimental 
plan  and  approach  allows  the  VI/HBR  results  to  be  utilized  in  two  important  ways: 

1)  The  results  can  be  replicated  and  validated  through  additional  work  outside  the  current  multi¬ 
national  research  groups;  and 

2)  The  experimental  process  can  be  used  to  implement  additional  national  research  that  will  be  of 
comparable  use  across  nations  by  incorporating  the  experimental  and  testing  process  enabled  by 
the  virtual  institute  prototype. 

The  experiment  demonstrates  the  utility  and  efficiency  to  be  gained  by  using  a  virtual  institute  and  its 
infrastructure  to  conduct  experiments. 
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Chapter  5  -  RESULTS  AND  LESSONS  IDENTIFIED 

Based  upon  the  meeting  results  and  investigations  conducted  by  the  SAS-053  team,  the  SAS-053  team 
recommends  establishment  of  a  Virtual  Institute  and  that  the  Virtual  Institute  should  produce  and  support  a 
Research  Plan  for  Team,  Group,  Organizational,  and  Cultural  Modeling  Research.  The  need  for  a  permanent 
VI  is  clear,  at  the  time  of  the  writing  of  this  report  (2007),  the  state  of  the  art  for  human  behavior,  cultural, 
group,  and  team  modeling  is  so  primitive  that  simulation  environments  can  only  achieve  realistic 
performance  for  only  very  small  numbers  of  automated  avatars  and  this  representation  is  relatively  primitive 
and  has  not  appreciably  advanced  in  capabilities  in  the  last  seven  years.  This  small  number,  typically  on  the 
order  of  tens  of  avatars  with  accurate  behaviors,  results  in  environments  that  do  not  accurately  portray  the 
real-world.  The  improvement  of  human  behavior  models  requires  the  incorporation  of  multiple  skill  levels 
and  representations  of  doctrine  and  they  must  reflect  societal,  training,  and  educational  differences.  Behavior 
moderators  such  as  morale,  fatigue,  stress,  fear,  information  operations,  and  intent  must  be  modeled  and 
included  within  the  model.  Furthermore,  the  knowledge  and  modeling  needed  to  be  able  to  accurately  model 
large  numbers  of  people  from  differing  cultures  is  not  available  and  will  not  become  available  unless 
experiments  at  large-scale  can  be  conducted  at  reasonable  cost.  The  virtual  institute  can  perform  the  required 
experiments  and  exercises. 


5.1  VALUE  OF  THE  VIRTUAL  INSTITUTE  CONCEPT 

Based  upon  research  and  analysis  performed  to  date,  the  following  claims  concerning  the  value  of  the 
virtual  institute  concept  can  be  substantiated: 

•  Cost-effective  development  of  human  behavior  models  for  individuals,  teams,  and  groups  that 
include  appropriate  behavior  modifiers; 

•  Accelerate  the  development,  use,  and  re-use  of  human  behavior  models  across  the  entire  spectrum 
of  NATO’s  need  for  the  models; 

•  Rapidly  address  technical,  human  behavior  modeling,  and  cultural  issues  of  interest  to  NATO  by 
rapidly  assembling  experts  from  across  alliance  to  address  the  problem; 

•  Reduce  the  cost  of  human  behavior  model  related  knowledge  acquisition; 

•  Rapidly  address  operational  issues  of  interest  to  NATO  by  rapidly  assembling  experts  and  military 
members  from  across  alliance  to  address  the  problem; 

•  Enable  cost-effective  use  of  multiple  experts  from  many  fields  to  address  a  problem; 

•  Enable  future  work  to  effectively  build  upon  prior  research  and  experience  where  leave-behind 
from  every  experiment  is  readily  accessible  to  all; 

•  Enable  re-use  and  expansion  of  existing  and  to  be  developed  human  behavior  models  by  developing 
standard  interfaces  and  middleware  to  HBR  models,  human  behavior  knowledgebases,  and  models 
that  portray  factors  that  affect  human  behavior  (such  as  culture,  teams,  and  experience); 

•  Provides  the  most  cost  effective  means  for  rapidly  addressing  the  need  for  higher  fidelity  human 
behavior  and  cultural  models; 

•  Provide  an  environment  for  test  and  evaluation  of  multi-cultural  and  multi-national  information 
assurance  and  software  protection  mechanisms  as  well  as  their  effect  upon  human  behavior, 
cultural-based  reactions,  team  building,  trust  establishment,  socialization,  and  other  factors  that 
affect  multi-national  operations; 

•  Provide  an  effective  foundation  for  conducting  research  on  human  behavior  model  verification, 
validation,  and  accreditation  (VV&A); 
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•  The  Virtual  Institute  offers  a  unique  combination  of  low  cost,  responsiveness,  broad  expertise, 
scientific  validity,  partnership/collaboration,  scalability,  access  to  results,  support  for  analysis, 
and  support  to  the  war-fighter  in  all  domains  of  concern,  not  just  human  behavior  modelling;  and 

•  Pioneer  and  enable  rapid  exploitation  of  emerging  Web  2.0  technologies,  such  as  wikis,  podcasts, 
telecasts  and  even  chatrooms  and  myspace-like  capabilities,  tied  to  the  VI  and  its  projects,  for  HBR 
research  of  all  types. 

In  short,  the  virtual  institute  concept  is  able  to  provide  a  broad  range  of  capabilities  that  are  required  by 
NATO  by  exploiting  a  large  set  of  the  powerful  Web  2.0  technologies  to  address  the  human  behavior  and 
cultural  modeling  research  issues  at  reasonable  cost  and  in  a  timely  manner. 

5.2  RECOMMENDED  ONGOING  VIRTUAL  INSTITUTE  ACTIVITIES 

The  goals  of  the  ongoing  activities  of  the  Virtual  Institute  are  to  push  current  technologies  and  research 
breakthroughs  to  participating  NATO  nations,  to  make  VI  research  and  exercise  results  readily  obtainable 
by  NATO  member  nations,  and  to  encourage  use  of  the  VI  by  researchers  to  conduct  multi-national 
experiments  in  HBR  and  other  areas  of  interest  to  NATO.  In  addition,  there  are  a  number  of  activities  that 
the  VI  should  conduct  as  part  of  its  mission  in  addition  to  hosting  experiments  and  exercises.  These 
activities  include  the  following: 

•  Develop  a  virtual  institute  technology  development  strategy; 

•  Develop  virtual  institute  metrics  and  success  criteria; 

•  Develop  a  virtual  institute  research  plan  and  strategy  (recorded  in  both  standards  and  a  VI  wiki); 

•  Determine  the  key  steps  for  advancing  the  virtual  institute  capabilities  and  the  key  technologies 
needed  by  the  virtual  institute  to  conduct  experiments; 

•  Develop  a  methodology  to  assess  technical  benefits  of  virtual  institute-based  experiments  and 
tests  as  compared  to  conducting  equivalent  experiments  and  tests  in  one  location; 

•  Develop  guidelines  for  technology  sharing  among  virtual  institute  participants  (recorded  in  both 
standards  and  a  VI  wiki); 

•  Develop  ontologies  for  human  behaviors  (recorded  in  both  standards  and  within  a  VI  wiki); 

•  Develop  and  employ  a  methodology  that  can  be  used  to  organize  a  human  behavior  model  library 
for  each  country  and  for  all  countries  as  well  as  enable  the  development  of  common  human 
behavior  models  and  model  components  (interfaces  can  be  documented  using  podcasts  and  wikis); 

•  Develop  and  maintain  a  sustained  presence  within  an  online  virtual  world,  such  as  Second  Life, 
for  collaboration,  information  exchange,  demonstrations,  and  hosting  of  virtual  conferences. 
This  same  presence  can  also  be  used  for  limited  experimentation  and  presentation  of  results; 

•  Develop  a  virtual  institute  human  behavior  model  experiment  plan; 

•  Conduct  an  ongoing  podcast  lecture  series  on  the  virtual  institute,  HBR,  and  other  VI  uses  and 
results;  and 

•  Maintain  wikis,  podcasts,  telecasts,  chatrooms,  and  myspace-like  capabilities  for  community  use 
and  for  use  by  each  experiment/exercise  that  are  also  linked  to  relevant  projects  in  other  Panels  in 
the  RTO. 

5.3  RESULTS 

•  Support  HFM-138: 

•  Experiment  Setup. 
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•  Support  SAS-050: 

•  Discussion  of  Follow  on  Work. 

•  Input  to  RTA  Setup: 

•  Use  of  WISE  Portal;  and 

•  Input  to  VI  Meeting. 

•  Input  to  RTB  Strategic  Plan. 


5.4  LESSONS  IDENTIFIED 

The  LTSS  on  Human  Behaviour  Representation  recommended  a  study  on  Virtual  Institute  for  Human 
Behaviour  Representation  in  2000  to  the  SAS  Panel.  In  2002  the  exploratory  team  ET.V  started  to  work 
on  a  more  detailed  study  proposal,  which  was  finished  in  2003.  The  study  on  the  Virtual  Institute  for 
Human  Behaviour  Representation  started  with  its  first  meeting  in  2004  and  had  its  last  meeting  in  2007. 
In  this  time  from  2000  to  2007  the  underlying  technology  for  virtual  institutes  developed  very  fast.  So  are 
now  high-speed  web  access  and  tools  for  collaboration  widely  available  and  affordable  even  for  private 
users.  Video,  groupware  and  shared  workspaces  are  now  quite  common  in  office  environments.  Web  2.0 
developments  (like  Apple’s  me.com,  MySpace,  Yahoo,  BlogSpot,  Wikipedia)  have  especially  an  impact 
on  the  younger  generation  and  made  the  Internet  more  interactive.  What  was  proposed  with  the  LTSS 
became  at  least  in  technology  terms  reality  just  by  itself,  it  didn’t  need  an  RTO  study.  But  nevertheless  the 
use  of  the  new  technology  and  its  embedding  in  the  right  organization  still  needs  concepts  from  the  RTO. 
But  RTO  should  think  of  ways  how  to  make  this  concept  development  or  guidance  for  use  of  new 
technology  faster  than  with  the  standard  procedures,  which  took  in  this  case  between  proposal  and  results 
7  years.  Regarding  the  use  of  IT  technology  an  established  virtual  institute  could  help  to  play  this  role. 

The  study  SAS-053  on  Virtual  Institute  for  Human  Behaviour  Representation  of  supported  by  5  nations  and 
ACT  (NC3A  participation).  The  two  Co-Chairs  of  the  study  (USA  and  NC3A)  had  a  very  limited  budget  for 
the  study  work.  During  the  study  they  gave  briefings  even  on  high  level  (Adm.  Bartoli)  to  ask  for  financial 
support  for  a  prototype  virtual  institute  to  do  experiments,  but  didn’t  get  the  support.  The  experience  from 
that  is  that  commitment  of  people  (and  the  study  team  consists  of  very  committed  people)  is  not  enough. 
The  nations  and  NATO  have  to  commit  funds  for  the  build  up  of  an  infrastructure  if  needed. 

The  team  members  in  4  of  the  5  participating  nations  changed  during  the  study  period.  These  changes  cost 
time  in  the  study  work.  It  would  be  good  if  the  study  team  could  stay  together  and  doesn’t  have  to  spend 
time  in  bringing  new  members  up  to  speed. 
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Chapter  6  -  RECOMMENDATIONS 

6.1  RESEARCH  RECOMMENDATIONS 

As  the  follow-on  to  the  work  of  SAS-053,  we  recommend  that  NATO  take  the  steps  necessary  required  to 
lay  the  foundation  for  making  the  virtual  institute  a  permanent  capability  that  is  available  for  all  NATO 
research  efforts.  Based  upon  work  done  to  date,  SAS-053  has  additional  recommendations  for  Virtual 
Institute  use  and  administration.  These  recommendations  follow. 

SAS-053  recommends  that  one  of  the  first  tasks  that  the  RTG  should  undertake  is  drafting  a  Memorandum  of 
Understanding  (MOU)  to  address  intellectual  property  rights,  government  interests,  funding,  cost  sharing, 
results  sharing,  handling  of  security  and  classification  issues,  management,  commercial  support  and  other 
issues  that  are  important  to  the  operation  of  a  virtual  institute.  The  MOU  is  an  important  first  step  in  the 
development  of  the  virtual  institute  concept  because  it  will  lay  the  groundwork  for  the  scope  of  the  virtual 
institute  and  insure  the  protection  of  the  technologies  made  available  to  the  virtual  institute  by  the 
participating  countries. 

The  VI  should  lead  in  the  development  of  a  technology  roadmap  for  the  technologies  that  the  institute  hosts 
and  for  the  HBMs  that  are  under  development,  especially  those  that  address  cultural  issues.  The  roadmap 
should  provide  a  review  of  where  we  stand  technologically  as  well  as  the  issues  that  the  NATO  nations  need 
to  address.  As  part  of  the  roadmap,  the  VI  should  propose  an  experiment  series  that  can  assess  the  utility  and 
sufficiency  of  current  HBM  data  and  models  needed  by  the  member  nations  and  assess  where  improved  data, 
research,  practices,  and  standards  are  needed.  The  VI  can  and  should  also  act  to  maintain  a  historical  record 
of  the  development  and  refinement  of  the  HBMs,  including  at  least  the  data,  experiments,  and  validation 
procedures  that  led  to  the  adoption  of  each  model.  New  reasoning  techniques,  potentially  hybrids  of  existing 
techniques  or  novel  techniques,  for  unmanned  avatars  (computer  generated  actors)  used  in  simulation 
environments  are  needed;  the  VI  should  lead  the  way  with  a  series  of  cross-cultural  experiments  that  assess 
the  value  of  reasoning  techniques  like  fuzzy  logic,  Bayesian  networks,  neural  networks,  intelligent  agents, 
and  case-based  reasoning  for  avatars  in  military  simulation  environments  and  determine  which 
circumstances  call  for  the  use  of  each  type  of  reasoning  approach.  The  virtual  institute  should  also  take  the 
lead  in  the  development  of  standards  and  practices  for  teleworkshops  and  teleconferences  as  a  means  for 
accelerating  research  and  technology  dissemination.  For  the  institute  is  to  maximize  its  effectiveness,  the  VI 
must  actively  seek  out  the  involvement  and  cooperation  of  national  modeling  and  simulation  organizations. 
The  national  organizations  can  aid  in  developing  the  technology  roadmap,  support  teleworkshops  and 
teleconferences,  cooperate  in  wikis,  podcasts,  and  webcasts  aimed  at  facilitating  discussion,  experiment  and 
exercise  definition,  and  describing  uses  and  outcomes  achieved  with  the  VI.  The  national  organizations  can 
also  aid  in  research  project  selection  and  proposal  development,  but  the  VI  can  serve  as  the  hub  and 
coordination  point  for  member  nations  activities. 

Based  on  our  experience  and  research  to  date,  the  SAS-053  proposes  the  following  three  additional 
recommendations  that  should  be  undertaken  to  Assemble,  Test,  and  Validate  an  operational,  long-term 
Virtual  Institute  and  thereby  demonstrate  its  utility  and  also  test  the  process  flow  and  organizational 
structure.  The  VI  would  execute  multiple  parallel  projects  simultaneously  and  fully  validate  the  VI  concept 
and  determine  the  utility  of  the  results  without  making  a  great  funding  or  manpower  investment.  As  a  first 
step,  we  recommend  the  development  of  a  common  ontology(s)  and  common  interface  between  models  for  a 
single  experiment  that  can  be  conducted  to  demonstrate  the  value  of  the  virtual  institute  concept  and  also 
show  that  valid  human  behavior  results  can  be  obtained  using  a  virtual  institute.  The  second  step,  much  as  in 
the  proposed  process  flow,  would  be  the  development  of  use  cases  in  order  to  demonstrate  the  utility  of  the 
virtual  institute  concept.  These  use  cases  would  be  derived  from  operational  needs  and  the  operational 
environment  and  would  be  written  to  address  an  operational  need  via  experimentation  using  the  virtual 
institute.  The  use  cases  should  be  tied  to  the  projects  proposed  by  the  participating  countries.  The  third  step 
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would  be  to  fully  develop  a  series  of  projects,  derived  from  the  use  cases,  which  can  be  used  to  demonstrate 
and  evaluate  the  complete  virtual  institute  process  and  organizational  structure.  When  these  demonstration 
projects  are  successful,  a  decision  can  be  made  concerning  whether  to  develop  and  deploy  an  ongoing 
complete  virtual  institute. 

The  virtual  institute  should  be  used  to  develop  improved  human  behavior  models  of  all  types  at  all  scales. 
Improvement  in  human  behavior  models  requires  research  on  the  incorporation  of  multiple  skill  levels  and 
representations  of  doctrine  and  they  must  reflect  societal,  training,  and  educational  differences.  Behavior 
moderators  such  as  morale,  fatigue,  stress,  fear,  information  operations,  and  intent  must  be  modeled  and 
included  within  the  human  behavior  model. 

Develop  and  maintain  a  presence  within  Second  Life  for  cultural  modeling  experimentation  and  training. 
Its  low  cost,  ubiquity,  flexibility,  and  ease  of  access  make  it  an  ideal  component  of  the  virtual  institute  for 
executing  any  unclassified  activities. 

A  further  use  for  the  VI  is  to  assess  alternative  command  post  infrastructures  in  order  to  determine  the 
infrastructure  that  yields  the  most  effective  network  centric  operational  force.  Within  a  VI  that  models 
network  centric  operations,  all  of  the  factors  that  influence  joint  military  operations,  such  as  armed  forces 
structures,  cyberwarfare,  doctrine  and  missions,  can  be  accurately  portrayed  to  derive  research  results  that 
are  applicable  to  real-world  network-centric  operational  conditions.  This  research  can  be  used  to  further 
other  command  post  research,  such  as  to  study  Network  Centric  Warfare  Command  and  Control  Issues 
and  develop  of  guidelines  for  conducting  research  to  address  issues  of  similar  scale  and  importance  to 
coalition  network  centric  operations.  Specifically,  in  order  to  come  to  grips  with  the  NCW  C2  challenges 
for  coalition  operations,  we  propose  conducting  foundational  work  to  identify  policy  and  technical  issues 
that  must  be  resolved  to  address  coalition  NCW  C2  issues  and  to  conduct  coalition  command  NCW  C2 
experiments  and  exercises.  The  research  will  determine  how  to  use  a  virtual  institute  to  address  the  policy 
and  technical  issues  impeding  coalition  NCW-based  command  and  control  and  will  also  identify 
experimental  structures  and  releasibility  policies  needed  in  order  to  conduct  international  experiments. 
Issues  that  will  be  addressed  include  identification  of  best  practices  for  use  of  the  institute  for  C2 
experiments  and  exercises,  identifying  initial  measures  of  effectiveness  that  indicate  the  effectiveness  of 
the  virtual  institute  for  addressing  coalition  NCW  C2  issues,  and  initially  assessing  the  ability  of  a  virtual 
institute  to  identify  issues  and  evaluate  solutions  for  coalition  C2  within  a  NCW  environment.  Underlying 
these  issues  is  the  need  to  learn  how  to  measure  the  effect  of  decisions  regarding  control  and  management1 
of  people  and  data  within  and  across  teams.  The  VI  can  be  used  for  the  following: 

1)  To  assess  types  of  experiments  and  exercises  that  can  be  conducted  using  a  virtual  institute  to 
study  coalition  NCW-based  command  and  control; 

2)  To  develop  and  describe  requirements,  desired  capabilities,  and  objectives  for  a  virtual  institute 
that  can  support  coalition  C2  exercises  and  experiments  in  a  NCW  environment; 

3)  To  identify  policy  issues  that  affect  the  ability  of  a  virtual  institute  to  perform  the  required 
experiments; 

4)  To  develop  guidelines  and  practices  for  exercises  and  experiments  that  can  be  used  to  assess  and 
measure  coalition  command  and  control  performance  in  a  heterogeneous  force; 

5)  To  identify  issues  and  evaluate  solutions  for  coalition  C2  within  a  NCW  environment  as  they  are 
related  to  peacekeeping  operations,  anti-terror  activity,  coordination  with  NGOs,  and  coalition 
NCW  C2  for  combat  operations; 

6)  For  model  validation  testing  conducted  via  multiple  trials  of  shared  international  human  behavior 
models;  and 


The  difficulties  encountered  in  control  and  management  of  resources  is  a  topic  receiving  more  attention  of  late;  see,  for  example,  “An  AI 
Planning-based  Tool  for  Scheduling  Satellite  Nominal  Operations”,  by  Moreno,  Borrajo,  and  Meziat,  AI  Magazine ,  Vol.  25,  No.  4,  Winter 
2004,  pp.  9-27. 
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7)  For  distribution  and  testing  of  human  performance  data  necessary  for  the  development  and  use  of 
human  performance  models  in  many  areas  of  military  operations,  and  for  large  multi-national 
experiments  that  address  cultural,  team,  group,  and  organizational  modeling. 

The  result  of  this  effort  would  be  a  code  of  best  practices  that  codifies  the  information  concerning  how  to  use 
the  virtual  institute,  how  to  setup  and  execute  experiments  in  the  VI,  how  policy  issues  should  be  addressed 
for  an  experiment,  which  legal  and  intellectual  property  issues  must  be  addressed  before  and  after  an 
experiment,  and  how  to  best  exploit  the  facilities  of  the  VI  for  to  achieve  an  experimental  objective. 
The  achievement  of  these  seven  objectives,  the  guides,  and  the  code  of  best  practices  should  not  be 
committed  only  to  paper  or  placed  on  the  web  but  should  be  augmented  by  wikis,  podcasts,  and  webcasts  so 
as  to  more  deeply  document  these  products  but  also  facilitate  and  capture  discussion  of  the  products. 

An  additional  recommendation  of  SAS-053  is  that  the  RTG  improve  cooperation  between  the  HFM,  1ST, 
and  SAS  Panels,  the  NMSG,  and  NC3A  to  support  RTG  in  its  development  and  demonstration  of  a 
broad-scale  NATO  virtual  institute  capability.  To  achieve  this  goal,  SAS-053  suggests  the  cooperation  and 
coordination  with  other  interested  (and  necessary)  Panels  to  participate  in  the  RTG  for  the  VI.  In  our  view, 
HFM  is  needed  for  the  for  the  HBR  technical  aspects,  1ST  is  needed  for  the  virtual  institute  infrastructure 
aspects,  and  SAS  is  needed  for  the  concept  definition,  integration,  and  exploration  aspects.  In  addition, 
work  must  be  performed  in  conjunction  with  SAS  for  operational  analyses  support,  and  NMSG  is  needed 
for  the  support  for  use  of  many  of  the  HBR  developments  in  the  M&S  arena. 

For  the  virtual  institute  to  operate  effectively,  its  cybersecurity  must  be  insured.  While  we  have  addressed 
some  of  these  issues,  more  remains  to  be  done.  A  major  question  that  must  be  addressed  is  the  cost  of 
protection  as  related  to  its  benefits.  Cost  can  be  broken  down  into  three  parts:  efficiency  cost  (what  is  the 
performance  penalty  of  a  technique  or  combination  of  techniques),  implementation  cost,  and  maintenance 
cost  (i.e.  what  maintenance  cost  does  the  technique  impose?)  Another  question  that  must  be  addressed  is 
how  to  assess  and  evaluate  the  costs  and  benefits  of  different  approaches  to  VI  security  in  a  multi-national 
setting  to  that  the  VI  can  make  informed  decisions  about  the  degree  of  protection  needed  based  upon  the 
sensitivity  of  the  experiments,  software,  and  the  costs  involved  in  applying  the  required  application 
techniques.  Another  major  research  question  is  software  protection  metrics  as  they  relate  to  the  virtual 
institute.  Some  metrics  that  must  be  refined  further  are  resilience  (a  measure  of  how  difficult  is  it  to  defeat 
a  technique),  obscurity  (a  measure  of  how  difficult  it  is  to  determine  if  a  particular  protection  technique 
has  been  employed),  and  expected  longevity  (a  measure  of  the  length  of  time  that  a  protection  technique 
will  afford  a  worthwhile  degree  of  protection)  of  each  protection  technology,  the  costs  and  benefits  of 
different  mixtures  of  protection  techniques,  and  the  level  of  protection  required  for  a  given  application  and 
simulation  environment  as  a  whole. 

Other  near-term  research  needs  regarding  cyber- security  for  the  VI  are  clear.  One  need  is  for  the 
development  of  standard  test  suites  that  can  be  used  for  evaluation  of  software  protection  methodologies  in 
a  scientific  manner  so  that  the  security  of  the  VI  can  be  verified  on  a  regular  basis.  Standard  testing  should 
be  coupled  with  the  development  of  data  protection  technology  assessment  standards,  standard  data  sets, 
and  methodologies  for  evaluating  data  protection  technologies.  A  further  need  is  for  the  development  of 
international  standards  to  promote  interoperability  and  trust  and  insure  that  systems  operate  at  equivalent 
levels  of  application/software  security  when  international  simulations  are  conducted.  There  are  other 
important  research  needs  as  well.  One  of  these  needs  is  a  methodology  that  can  be  used  to  determine,  in  a 
standard  manner,  the  degree  of  protection  required  for  the  VI  for  a  given  purpose  and  for  its  simulation 
environment  as  a  whole.  A  further  need  is  for  the  development  of  standard  attack  profiles  that  are  coupled 
with  intelligent  agents  in  order  to  enable  rapid,  autonomous  evaluation  of  the  effectiveness  of  a  defence 
within  a  distributed  simulation  environment  (as  suggested  by  Shrobe).  Of  course,  the  multi-cultural  and 
multi-national  information  assurance  and  software  protection  mechanisms  as  well  as  their  effect  upon 
human  behavior,  cultural-based  reactions,  team  building,  trust  establishment,  socialization,  and  other 
factors  that  affect  multi-national  operations  must  be  investigated. 
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We  believe  that  the  utility  of  the  virtual  institute  and  human  behavior  modeling  would  be  highlighted  and 
interest  in  these  technologies  would  be  increased  if  the  next  Panel  focused  on  the  development  of  a  tool  as 
a  result  of  the  Panel’s  efforts.  Our  suggestion  for  the  tool  is  the  development  of  a  multi-national,  coalition 
submarine  warfare  simulation  environment.  This  environment  would  provide  opportunities  for  cultural 
and  team  research,  modeling,  and  simulation  within  each  submarine  as  well  as  for  cross-cultural  and 
multi-national  modeling  and  simulation  in  the  command  and  control  of  submarine  operations.  Given  the 
increasing  importance  of  submarines  to  the  control  of  littoral  areas  and  maritime  chokepoints  as  well  as 
for  the  protection  of  surface  ship  battlegroups  and  supply  ships,  a  successful  tool  would  insure  its  adoption 
and  use  as  well  as  interest  in  further  development  of  the  virtual  institute  concept  and  interest  in  human 
behavior  research. 

One  area  of  clear  importance  to  the  Virtual  Institute  that  remains  to  be  addressed  is  increased  participation 
by  the  member  countries  in  various  projects,  which  can  be  accomplished  by  a  series  of  capabilities 
briefings  delivered  at  key  meetings  and  via  the  internet.  Additional  projects  from  each  participating 
country  will  be  the  foundation  for  the  exploration  of  the  complete  NATO  VI  concept,  which  will  lead  to  a 
recommended  design  and  development  process  for  a  VI  for  HBR  and  for  virtual  institutes  for  the  other  six 
Panels. 

The  study  team  discussed  recommendations  to  the  RTO  for  follow  on  work  in  the  area  of  human  behaviour 
representation  and  the  use  of  the  virtual  institute  technology.  The  following  recommendations  are  given  in 
the  priority  the  team  gave  to  them  (highest  priority  first). 


6.2  SAS  RSG  -  SETUP  AND  ADMINISTRATION  OF  HBR  VI  BY  RTA  IS 

6.2.1  Description 

A  virtual  institute  for  modelling  military-related  individuals,  teams,  groups,  platforms,  and  organizations 
must  be  established  creating  a  Web-based  clearing  house  of  databases,  models  and  model  components, 
standards,  and  requirements  to  support  multi-national  research  and  development  on  human  behaviour 
modelling.  A  Wiki  for  HBR  and  a  MySpace  for  the  HBR  community  should  be  provided.  The  technical  IT 
support  staff  of  the  virtual  institute  could  be  RTA  IS.  The  research  support  staff  and  the  lead  of  the  virtual 
institute  with  the  scientific  oversight  should  come  from  the  nations. 

6.2.2  Military  Benefits 

The  virtual  institute  will  provide  an  essential  mechanism  for  information  exchange  across  the  NATO 
nations,  reducing  duplication  of  effort  and  maximizing  national  capabilities  in  the  domain  of  human 
behaviour  modelling. 

6.2.3  Justification 

The  virtual  institute  for  modelling  human  behaviours  is  called  for  in  the  NATO  M&S  Master  Plan  (3.3.3: 
exchange  of  information). 

6.2.4  Short  and  Long  Term  Outcome 

Short  term:  The  short  term  outcome  will  be  the  creation  of  a  Web-based  clearing  house  for  information 
related  to  human  behaviour  modelling  for  military  applications. 

Long  term:  The  long  term  outcome  will  be  the  multi-national  capability  for  reduced  build-time  costs  in 
human  behaviour  modelling. 
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6.2.5  Required  Resources 

NATO  RTA  for  virtual  institute  infrastructure  and  technical  IT  support  staff  and  national  commitment  and 
resources  for  virtual  institute  infrastructure  on  their  side  and  research  support  staff. 

6.3  HFM  RSG  -  HER  TEAM  AND  ORGANIZATION  ONTOLOGY 
DEVELOPMENT 

6.4  HFM  RSG  -  MULTI-NATIONAL  EXPERIMENT  HFM-138  FOLLOW  ON 

6.5  HFM  RSG  -  ADOPT  EXISTING  TECHNOLOGY  IN  DEVELOPING 
PODCAST  LECTURE  SERIES  FOR  RTA 

6.5.1  Description 

Many  universities  and  research  institutes  use  already  podcast  technology  to  deliver  lectures  on  the  Internet 
(e.g.  on  iTunes).  NATO  should  use  this  technology  too  to  make  existing  lecture  series  and  new  lecture  series 
available  for  the  interested  public.  Lectures  with  unlimited  distribution  could  be  placed  on  the  RTA  web  site 
whereas  classified  material  could  be  placed  on  the  RTA  web  side’s  closed  area  (NATO  Unclassified)  or  on 
the  NATO  Secret  network.  From  the  human  factors  community  the  layout  of  such  podcasts  and  the  setup  of 
a  facility  to  cover  the  lectures  should  be  supported. 

6.5.2  Military  Benefits 

The  availability  of  lecture  series  will  foster  a  better  information  exchange  and  help  reducing  duplication  of 
effort  and  maximizing  national  capabilities  in  the  research  domain. 

6.5.3  Justification 

Lecture  series  are  part  of  the  RTO  procedures  and  intended  to  distribute  research  results  to  all  participating 
nations.  A  better  distribution  of  lecture  series  to  the  research  community  is  a  goal  of  the  RTO. 

6.5.4  Short  and  Long  Term  Outcome 

Short  term:  The  short  term  outcome  will  be  a  lecture  series  which  can  be  delivered  on  the  Internet. 

Long  term:  The  long  term  outcome  will  be  the  setup  of  a  facility  to  produce  podcasts  for  lecture  series  and 
procedures  based  on  a  human  factors  analysis. 

6.5.5  Required  Resources 

NATO  RTA  for  technical  infrastructure  and  national  commitment  and  resources. 

6.6  SAS  RSG  -  DATA  EXCHANGE  PRACTISES  IN  NATO 

6.6.1  Description 

The  work  on  the  study  showed  that  no  overview  on  data  exchange  practices  in  NATO  exists.  A  study 
should  be  started  to  provide  the  research  community  with  an  overview  on  the  issue  and  a  list  of  resources 
where  respective  information  could  be  found  in  the  nations  and  in  NATO.  The  study  should  also 
investigate  into  the  opportunity  in  building  up  an  organizational  element  where  this  information  can  be 
stored  and  maintained. 
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6.6.2  Military  Benefits 

The  availability  of  an  overview  on  data  exchange  practices  and  a  reference  on  agreements  will  foster  a 
better  information  exchange  and  help  reducing  duplication  of  effort  and  maximizing  national  capabilities 
in  the  research  domain. 

6.6.3  Justification 

Data  exchange  agreements  are  necessary  for  multi-national  research. 

6.6.4  Short  and  Long  Term  Outcome 

Short  term:  The  short  term  outcome  will  be  an  overview  on  the  issues  and  the  identification  of  an 
organizational  element  to  be  build  up  for  the  future  to  store  and  maintain  a  reference. 

Long  term:  The  long  term  outcome  will  be  an  organizational  element. 

6.6.5  Required  Resources 

NATO  and  national  commitment  and  resources. 


6.7  HFM  RSG  -  THE  USE  OF  VIRTUAL  WORLDS  FOR  NATO  R&D  IN  HBR 
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Annex  A  -  ET.V  FINAL  REPORT 

NATO  STUDIES,  ANALYSIS,  AND  SIMULATION 
(SAS)  PANEL  ADVISORY  REPORT 

A  VIRTUAL  INSTITUTE  (VI)  FOR  RESEARCH  ON 
HUMAN  BEHAVIOR  REPRESENTATION  (HBR) 

NATO  SAS  PANEL  EXPLORATORY  TEAM  ET.V  -  FINAL  REPORT 

EXECUTIVE  SUMMARY 

The  topic  and  focus  of  the  SAS-ET.V  exploratory  team  was  derived  from  a  previous  NATO  Long  Term 
Scientific  Study  (LTSS/51)  on  Human  Behavior  Representation  sponsored  by  the  SAS  Panel.  The  purpose  of 
the  LTSS  was  to  provide  a  report  for  the  use  by  NATO  and  national  authorities  on  the  implications  of 
technological  developments  to  military  operations  over  the  next  ten  to  fifteen  years  and  to  provide  research 
planners  with  recommendations  for  research  that  address/investigate  the  implications  of  projected 
technological  developments.  Of  particular  interest  to  this  project,  under  LTSS/51,  their  recommendation  of 
highest  priority  was  to  establish  a  NATO  RTO  exploratory  team  to  investigate  the  feasibility  and  utility  of 
assembling  a  virtual  institute  for  research  on  human  behavior  modeling  (NATO,  2001).  Specifically, 
the  virtual  institute  was  to  be  established  for  the  purpose  of  modeling  military-related  individuals,  teams, 
groups,  platforms,  and  organizations  in  their  performance  of  military  operations  and  tasks.  The  virtual 
institute  was  envisioned  as  a  NATO  resource  that  would  accelerate  technology  dissemination  in  specific 
technology  areas  that  would  include  development  of  a  web-based  clearing  house  of  databases,  models  and 
model  components  and  developing  standards,  and  requirements  to  support  multi-national  research  on  human 
behavior  modeling.  This  particular  LTSS/51  recommendation  was  presented  to  and  approved  by  the  SAS 
Panel,  which  in  turn  established  SAS-ET.V.  The  objective  of  this  SAS-ET.V  was  two-fold.  Lirstly, 
SAS-ET.V  is  to  advise  the  SAS  Panel  and,  through  cooperation  with  other  RTO  Panels,  the  RTO  on  the 
technical  merit  and  feasibility  of  establishing  a  virtual  institute  within  NATO.  Secondly,  the  SAS-ET.V  is  to 
advise  the  SAS  Panel  and,  through  cooperation  with  other  RTO  Panels,  the  RTO  the  feasibility  of  enabling 
multi-national  research  and  development  in  human  behavior  modeling  by  using  the  virtual  institute  concept. 

To  address  the  two  objectives  for  the  Panel,  several  different  activities  were  undertaken  during  the  past 
year  to  assess  the  feasibility  and  desirability  of  the  two  concepts.  As  a  first  step,  an  exploratory  team  was 
formed  and  three  meetings  were  held  to  discuss  technical  feasibility  of  the  concepts,  to  conduct  technical 
interchange  and  to  assess  NATO-wide  support  for  the  concepts.  In  addition,  team  members  participated  in 
their  individual  countries’  technical  activities  related  to  the  two  concepts  in  order  to  assess  their  own 
countries’  technical  experts’  opinions  concerning  the  concepts  as  well  as  support  for  the  concepts  and 
commitment  to  their  execution.  A  final  area  of  endeavor  was  an  ongoing  assessment  of  the  technological 
foundations  needed  to  implement  and  operate  a  virtual  institute.  In  brief,  there  is  support  for  the  concepts; 
the  technologies  needed  to  implement  the  concepts  are  available  and  have  been,  in  many  cases, 
demonstrated;  and  there  is  widespread  belief  that  the  NATO  community  would  benefit  from  the  virtual 
institute  concept  and  that  human  behavior  representation  is  an  appropriate  choice  for  the  demonstration  of 
the  virtual  institute  concept. 

In  light  of  the  research,  recommendations  made  by  colleagues,  and  other  investigations  conducted  over  the 
past  year,  the  SAS-ET.V  team  recommends  that  a  RTO  Task  Group  (RTG)  be  established  to  develop  a 
demonstration  Virtual  Institute  and  that  the  Virtual  Institute  should  produce  a  Research  Plan  for  Team, 
Group,  Organizational,  and  Cultural  Modeling  Research. 
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A.l  INTRODUCTION 

The  continuing  acceleration  of  the  rate  of  change  of  technology  development  and  the  ever  increasing 
pressure  to  maximize  the  benefit  of  scarce  military  resources  point  to  a  need  to  change  the  manner  in  which 
NATO  conducts  research  and  disseminates  research  results  for  exploitation.  While  in  the  past  activities  such 
as  conferences,  workshops,  and  meetings  have  served  the  community  well  in  organizing  and  disseminating 
information,  they  no  longer  are  sufficient  to  allow  scientists  and  decision-makers  to  remain  appraised  of 
relevant  scientific  developments  and  so  a  new  approach  to  managing,  conducting,  and  disseminating 
research  that  augments  traditional  means  is  indicated.  At  this  time,  it  is  clear  that  a  capability  is  needed  that 
supports  the  rapid  formation  of  ad  hoc,  multi-disciplinary  research  teams,  accelerates  their  work  by  making 
prior  work  and  related  expertise  readily  available,  fosters  rapid  dissemination  of  research  results, 
and  otherwise  aids  in  the  execution  and  exploitation  of  relevant  research.  Because  of  the  academic  nature  of 
the  activities  to  be  conducted  and  because  the  organization  would  only  exist  within  cyberspace,  this 
capability  is  called  a  virtual  institute.  The  technologies  underpinnings  needed  to  implement  a  virtual 
institute  have  only  recently  become  available  at  a  cost  and  performance  that  makes  their  use  efficient  and 
effective  for  all  NATO  members.  In  order  to  make  the  virtual  institute  an  integral  part  of  the  NATO  research 
structure,  all  that  must  be  done  is  to  bring  the  foundational  technologies  together  in  a  manner  conducive  to 
matching  the  information  dissemination  capability  with  the  research  execution  capability  and  to  demonstrate 
the  technical  feasibility  and  savings  that  would  accrue  due  to  the  proper  exploitation  of  available 
technologies.  After  a  year  of  study  and  effort,  the  structure  and  operation  of  the  virtual  institute  is  apparent 
and  the  team  strongly  encourages  execution  of  a  proof  of  concept  demonstration  in  the  human  behavior 
representation  (HBR)  arena  within  the  RTO  Task  Group  framework.  The  remainder  of  this  report  documents 
how  the  exploratory  team  arrived  at  its  conclusions,  the  basis  for  its  conclusions,  and  its  suggestions  for  the 
organization  and  operation  of  the  virtual  institute. 

The  topic  and  focus  of  the  current  SAS-ET.V  exploratory  team  was  derived  from  a  previous  NATO  Long 
Term  Scientific  Study  (LTSS/51)  on  Human  Behavior  Representation  sponsored  by  the  SAS  Panel. 
The  purpose  of  the  LTSS  was  to  provide  a  report  for  the  use  by  NATO  and  national  authorities  on  the 
implications  of  technological  developments  to  military  operations  over  the  next  ten  to  fifteen  years  and  to 
provide  research  planners  with  recommendations  for  research  that  address/investigate  the  implications  of 
projected  technological  developments.  Of  particular  interest  to  this  project,  under  LTSS/51,  their 
recommendation  of  highest  priority  was  to  establish  a  NATO  RTO  exploratory  team  to  investigate 
the  feasibility  and  utility  of  assembling  a  virtual  institute  for  research  on  human  behavior  modeling 
(NATO,  2001).  Specifically,  if  implemented  the  virtual  institute  was  to  be  established  for  the  purpose  of 
modeling  military -related  individuals,  teams,  groups,  platforms,  and  organizations  in  their  performance  of 
military  operations  and  tasks.  The  virtual  institute  project’s  work  included  creating  a  web-based  clearing 
house  of  databases,  models  and  model  components  and  developing  standards,  and  requirements  to  support 
multi-national  research  on  human  behavior  modeling.  This  particular  LTSS/51  recommendation  was 
presented  to  and  approved  by  the  SAS  Panel,  which  in  turn  established  SAS-ET.V.  The  objective  of  the 
SAS-ET.V  is  two-fold.  Lirstly,  SAS-ET.V  is  to  advise  the  SAS  Panel  and,  through  cooperation  with  other 
RTO  Panels,  the  RTO  on  the  technical  merit  and  feasibility  of  establishing  a  virtual  institute  within 
NATO.  Secondly,  the  SAS-ET.V  is  to  advise  the  SAS  Panel  and,  through  cooperation  with  other  RTO 
Panels,  the  RTO  the  feasibility  of  enabling  multi-national  research  and  development  in  human  behavior 
modeling  by  using  the  virtual  institute  concept. 

A.1.1  Definition  of  a  Virtual  Institute 

After  starting  the  process  of  defining  the  capabilities  required  to  execute  distant  research  programs  and  to 
serve  as  a  repository  of  research  results  and  knowledge,  the  team  determined  that  a  formal  name  for  the 
type  of  proposed  virtual  organization  was  needed.  After  some  discussion,  the  term  institute  was  selected 
because  of  its  positive  connotations  and  its  research  orientation.  Additionally,  the  term  institute  is  a  richer 
descriptive  term  than  alternative  terms,  such  as  laboratory,  testbed,  or  workshop  and  also  has  more  of  a 


A  -  2 


RT  O-TR-S  AS-053 


NATO 

OTAN 


ANNEX  A  -  ET.V  FINAL  REPORT 


connotation  of  being  distributed.  The  term  institute  also  has  more  of  connotation  of  support  for  broadly 
scoped  research  investigations  than  the  term  laboratory,  which  generally  has  a  connotation  of  a  narrower 
research  focus.  The  term  testbed  lacks  the  connotation  of  scope  and  permanence  of  organization  than 
institute,  an  institute  generally  has  a  long  lifetime  and  connotes  some  sort  of  permanence,  whereas  a 
testbed  exists  until  the  end  of  its  test  function  and  then  is  dismantled.  In  the  team’s  view,  a  testbed  would 
be  a  natural  component  of  an  institute.  By  using  the  term  institute  a  number  of  other  components  of  the 
organization  could  be  inferred  that  would  not  even  be  implied  by  the  use  of  the  words  testbed  or 
laboratory;  some  of  these  additional  components  of  the  organization  would  be  a  library  and  clearing 
house.  Calling  the  product  an  institute  is  the  consensus  of  ET.V  and  also  seems  to  be  the  dominant  choice 
of  the  colleagues  with  whom  we  have  discussed  the  project. 

A.1.2  Capabilities/Functionality  Requirements  for  a  Virtual  Institute 

To  provide  the  breadth  of  services  needed  by  institute  users,  the  institute  would  require  a  number  of 
capabilities.  The  institute  should  be  able  to  provide  training,  electronic  library,  news  center,  and  host  for 
exercises/demonstrations.  The  institute  would  focus  on  addressing  new  areas  of  need  for  NATO,  such  as 
crowd  modeling,  cultural  modeling,  and  joint  training  (standard  military  applications).  Some  example 
capabilities  that  should  be  provided  by  the  virtual  institute  and  must  be  developed  within  an  international 
venue  would  be  to  serve  as  a  clearing-house  for  research  results,  library,  study  coordination,  testbed 
hosting,  and  library.  A  prime  function  would  be  distribution  of  validated  and  nation- specific  human 
performance  models  to  NATO  members  and  organizations  that  need  them.  An  associated  required 
capability  is  the  ability  for  promulgating  information  concerning  the  availability  of  models  and  their 
characteristics  to  member  nations  and  national  organizations.  In  addition,  the  institute  must  be  able  to 
provide  an  overview  and  insight  into  ongoing  academic  research,  the  theoretical  and  experimental 
underpinnings  for  the  different  models  that  it  provides,  and  advice  and  theoretical,  as  well  as  practical, 
justification  for  using  certain  models  within  a  given  simulation  or  real  world  environments. 

The  virtual  institute  could  also  vitalize  the  development  of  cross-national  academic  research  support  for 
model  development  and  support  for  model  validation.  More  importantly,  because  it  is  virtual,  the  institute 
would  be  able  to  support  multiple  parallel  research  efforts  and  assist  the  efforts  and  model  development  in 
general  by  providing  facilities  and  access  to  researchers  to  support  and  energize  high-bandwidth  exchange 
and  comment  about  data  and  experimental  programs  between  researchers  and  the  military  community  during 
experimentation.  The  institute  would  help  researchers  to  maximize  the  results  of  their  efforts  and  the  utility 
and  relevance  of  their  research  results  by  giving  them  ready  access  to  the  research  results  of  parallel,  ongoing 
efforts  and  supporting  commentary  and  collegial  interchange  about  the  research  in  real-time.  The  virtual 
institute  would  thereby  maximize  the  return  on  research  investment  by  allowing  researchers  to  incorporate 
and  exploit  for  their  own  research  other  relevant  results  from  any  member  nation  as  it  is  produced. 
This  exploitation  could  occur  while  the  work  is  ongoing  and  thereby  allow  researchers  to  test  and  validate 
parallel  research  results  during  the  course  of  their  own  experimentation.  The  institute  would  also  allow 
developers  to  readily  access  researchers  whose  work  is  relevant  to  their  development  and  obtain  their 
assistance  and  guidance  in  using  the  results  of  their  research  in  the  development  project.  The  virtual  institute 
would  allow  the  researchers  to  accomplish  research  related  activities,  such  as  validation  and  peer  review,  in  a 
few  days  or  hours  instead  of  months;  thereby  accelerating  the  rate  of  progress  on  research,  development 
based  on  research  results,  and  the  transition  of  research  results  into  the  operational  community.  In  short, 
a  virtual  institute  could  dramatically  shorten  research  and  development  time  and  shorten  the  length  of  time 
that  lapses  between  discovery  and  operational  use. 

Another  function  of  the  virtual  institute  would  be  to  provide  cross-referencing  and  linkage  services  between 
academic  literature  and  models  developed  at  the  institute  and  in  the  individual  nations.  The  cross-referencing 
would  take  the  form  of  a  semantic  web  to  support  research  and  development  by  forming  links  between 
models  stored  and  indexed  at  the  institute  and  the  supporting  research/literature  for  the  models.  Another 
function  that  the  institute  can  fulfill  is  to  disseminate  the  information  and  data  developed  via  research 
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projects  via  the  web,  institute  databases,  and  virtual  conferences.  Another  way  to  exploit  the  institute  and  its 
capabilities  may  be  to  determine  how  discrete  value  systems  influence  behaviours  across  ethnicities  and 
other  societal  groups.  This  issue  is  an  important  research  question  with  practical  military  implications  and 
applications  for  the  results  of  the  research,  but  it  has  not  been  undertaken  to  date  because  of  the  cause  of 
assembling  the  requisite  international  team.  The  institute  can  take  the  lead  in  addressing  this  important  issue 
since  it  need  not  bring  all  of  the  researchers  to  one  location  for  them  to  perform  their  work,  rather  they  will 
be  able  to  remain  in  their  own  labs  and  develop  an  international  joint  experimental  plan  to  address  the 
research  issue.  The  institute  can  also  take  the  lead  in  developing  standard  ways  for  describing  and  interfacing 
to  computerized  models  so  that  cross-linkages  between  models  and  their  supporting  data  and  source 
authorities  are  established  and  so  that  the  models  and  the  linkages  are  expandable. 

The  institute  should  also  be  poised  to  lead  in  the  development  of  a  technology  roadmap  for  the  technologies 
that  the  institute  hosts.  The  roadmap  can  provide  a  review  of  where  we  stand  technologically  as  well  as 
where  we  need  to  go.  As  part  of  the  roadmap,  the  institute  can  assess  where  we  have  data  and  models  needed 
by  the  member  nations  and  where  we  need  data  as  well  as  where  we  need  standards.  The  institute  can  and 
should  also  form  some  sort  of  historical  record  so  that  there  is  a  record  of  how  the  models  were  developed 
and  the  data  were  acquired  as  well  as  how  the  models  were  validated.  The  Virtual  Institute  should  also 
support  collaboration  and  virtual  meeting  support  and  support  distance  learning  with  the  goal  of  providing 
standard  meetings/open  forums  at  traditional  workshops  and  conferences  such  as  BRIMS,  ICCM,  and  other 
conferences  in  order  to  engage  the  research  and  military  application  communities  and  make  them  aware  of 
the  institute,  its  abilities,  accomplishments,  and  community  needs.  The  virtual  institute  should  also  take  the 
lead  in  the  development  of  teleworkshops  and  teleconferences  as  a  means  for  accelerating  research  and 
technology  dissemination  and  to  minimize  the  infrastructure  required  to  host  and  operate  a  workshop  or 
conference.  In  our  view,  if  the  institute  is  to  maximize  its  effectiveness,  the  institute  must  actively  seek  out 
the  involvement  and  cooperation  of  national  modeling  and  simulation  organizations.  The  national 
organizations  can  aid  in  developing  the  technology  roadmap,  support  teleworkshops  and  teleconferences, 
and  aid  in  research  project  selection  and  proposal  development.  A  further  important  function  of  the 
participating  nations  is  to  provide  funding  to  the  institute. 

One  topic  that  must  be  addressed  in  detail  in  the  operation  of  the  institute  given  its  reliance  upon 
networking  capabilities  is  network  and  application  software  security.  While  improvement  of  security 
capabilities  in  these  technological  areas  is  ongoing  among  many  of  the  member  nations,  the  institute  must 
address  data,  application,  and  network  security  issues  within  its  own  context  by  drawing  upon  member 
nations  capabilities  and  research  results  so  that  it  to  operate  effectively  and  securely. 

A.1.3  Application  Focus  for  a  Virtual  Institute:  HBR 

As  an  initial  research  arena  to  demonstrate  the  value  and  viability  of  the  virtual  institute  concept,  SAS- 
ET.V  recommends  human  behavior  representation  (HBR).  The  group’s  examination  of  major  HBR 
conferences  and  workshops  showed  little  evidence  of  cross-national  or  multi-national  cooperation  and 
research  efforts  in  HBR,  as  a  result  most  HBR  results  are  nation  and  culture  specific.  The  proposed  virtual 
institute  would  foster  and  broaden  multi-national  cooperation  and  foster  development  of  multi-national 
and  cross-cultural  human  behavior  models.  As  a  result  of  the  virtual  institute’s  ability  to  foster  and 
accelerate  international  cooperation  and  research  teaming,  within  HBR  the  research  focus  would  be  on  the 
development  of  representations  for  human  behaviours  that  are  relevant  to  military  simulations  and 
operations  across  the  nations.  The  charter  for  the  virtual  institute  investigations  into  HBR  would 
encompass  the  modeling  of  model  team,  group,  and  organizational  behaviours  in  light  of  different 
influences  that  come  to  bear  on  their  behavior  based  upon  a  number  of  factors.  A  few  of  the  factors  that 
have  been  identified  and  that  are  in  need  of  research  and  development  are  the  effect  of  societal  groups: 
cultural,  religious,  and  ethnicity  as  part  of  and  modifiers  to  the  HBR.  A  central  research  issue  that  the 
virtual  institute’s  HBR  research  must  address  is  whether  to  consider  the  group  as  a  set  of  individuals  that 
come  together  to  form  a  group  or  whether  to  model  the  group  an  atomic  unit  that  can  possibly  decompose 
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into  individuals  when  necessary.  The  virtual  institute  can  play  a  key  role  here  because  it  can  foster 
international  research  into  these  topics  as  well  as  help  investigators  to  determine  the  effect  of  culture  and 
nationality  on  these  issues.  Research  also  needs  to  be  conducted  to  identify  the  different  crowd  types  to  be 
modeled.  Another  final  research  arena  that  has  been  identified  for  the  virtual  institute  is  the  development 
of  quantitative  and  qualitative  methods  for  the  representation  of  the  groups  and  their  responses.  A  final 
research  arena  that  should  be  a  key  virtual  institute  research  topic  is  investigation  of  team  situational 
awareness  as  a  component  of  HBR. 

Within  the  HBR  research  focus  arena  for  the  virtual  institute,  we  believe  that  the  guiding  principle  should 
be  the  identification  of  applications  that  have  high  relevance  to  military  missions  and  needs,  the 
applications  should  have  a  clearly  identified  need  and  would  be  operationally  helpful  and  relevant  across 
the  nations.  With  this  focus,  the  virtual  institute  would  be  able  to  insure  that  its  research  would  be  directly 
applicable  to  military  development  and  operational  areas  across  the  nations.  For  example,  one  clear  arena 
of  relevance  is  the  C4I  application  area,  in  this  arena  the  institute  could  be  used  to  allow  the  research 
results  to  service  the  operational  need.  Another  important  research  focus  arena  with  high  operational 
relevance  is  special  operations  forces;  especially  the  response  of  different  cultures  to  information 
operations.  As  such,  the  cultural  modeling  aspects  of  the  virtual  institute’s  HBR  research  would  be  of 
critical  importance  in  order  to  develop  and  maintain  a  useful  model  of  the  response  of  different  cultures  to 
information  warfare  and  other  non-traditional  types  of  warfare.  The  institute  would  provide  a  natural 
setting  for  cultural  modeling  experiments  since  it  would  provide  ready  access  to  a  wide  variety  of  cultures 
at  minimal  cost.  A  final  research  and  application  area  for  the  virtual  institute  would  be  negotiations,  both 
in  a  formal  setting  as  well  as  in  ad  hoc  settings  in  operational,  deployed  environments,  and  the  effects  of 
various  components  and  influences  of  HBR  on  negotiation  strategies  and  tactics.  As  a  result  of  the 
multitude  of  international  research  efforts  that  the  institute  could  foster  and  collate  in  a  manner  heretofore 
unachievable,  one  other  function  of  the  institute  that  would  valuable  would  be  the  development  of 
composable  HBR  data  models  and  means  for  storing  data  in  a  standard  manner  so  that  it  can  be  reused 
across  experiments.  In  addition  to  having  a  standard  baseline  model,  the  model  would  also,  because  of  the 
transnational  focus  of  the  institute,  require  the  capability  to  accept  culture  and  national  specific  modifiers 
on  behavior  as  “plug-ins”  to  the  baseline  behavior  model.  In  any  event,  the  virtual  institute  would  certainly 
be  in  a  position  to  play  a  key  role  in  the  development,  evaluation,  and  validation  of  an  international 
baseline  human  behavior  model  as  well  as  with  identifying  and  developing  important  culture  and  nation 
specific  factors  that  come  to  bear  upon  the  baseline  model. 


A.2  ET.V  OPERATION  AND  DOCUMENTATION 
A.2.1  Participating  Nations/Organizations 

The  following  countries  and  organizations  are  participating  or  have  expressed  an  interest  in  participating 
in  the  SAS-ET.V  effort  and  the  possible  follow-on  work  in  the  development  of  an  HBR  Virtual  Institute: 

•  Germany; 

•  United  Kingdom; 

•  Turkey; 

•  Belgium; 

•  United  States; 

•  Netherlands; 

•  Canada; 

•  NC3A;  and 

•  ACT. 
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A.2.2  Focus  Area  of  NATO  ET.V 

The  SAS-ET.V  has  a  refined  a  preliminary  multi-national  concept  for  a  NATO  RTO  Virtual  Institute  for 
HBR.  This  concept  includes  a  description  of  products  the  institute  should  provide,  a  description  of 
resources  needed  by  the  virtual  institute,  the  development  of  an  organizational  structure  and  architecture 
for  the  virtual  institute,  and  a  concept  of  operation  for  the  virtual  institute,  including  virtual  and  real 
meetings.  SAS-ET.V  is  composed  of  experts  in  the  two  central  technical  issues  faced  by  the  exploratory 
team: 

1)  The  creation  of  a  web-based/network-based  virtual  institute;  and 

2)  The  institute’s  initial  application  on  human  behavior  modeling. 

The  responsibilities  of  the  current  ET.V  team  are  to  inventory  and  characterize  the  team,  group  and 
organizational  models  currently  available  (taken  directly  from  the  LTSS/51  effort),  define  requirements 
for  models  in  this  domain  considering  the  range  of  applications  of  potential  interest  across  the  NATO 
nations,  and  to  create  a  research  roadmap  to  address  identified  shortfalls  and  opportunities.  These  tasks 
will  be  accomplished  using  the  demonstrator  Virtual  Institute.  It  is  clear  at  this  point  in  the  team’s  progress 
that  to  support  the  research,  development,  and  validation  of  models  needed  to  make  current  NATO 
military  needs  a  reality,  a  multi-national  effort  in  the  understanding  and  modeling  of  human  behavior  and 
the  sharing  of  the  modeling  results  across  nations  is  needed.  It  is  also  apparent  from  the  team’s  efforts  to 
date  that  the  development  of  a  virtual  institute  for  research  in  human  behavior  representation  (HBR)  will 
enable  the  multi-nation  cooperation  necessary  for  modeling  human  behavior  to  address  future  modeling 
and  simulation  requirements  and  may  be  able  to  minimize  costs. 

Since  the  field  of  human  behavior  representation  is  so  broad,  the  SAS-ET.V  was  chartered  by  the  SAS 
Panel  to  investigate  a  few  specific  areas  of  HBR  using  the  virtual  institute  concept  and  to  evaluate  the 
virtual  institute  concept  within  these  specific  areas  of  HBR.  A  main  focus  is  on  the  development  of 
representations  for  human  behaviors  that  are  relevant  to  military  simulations  and  operations,  including 
team,  group,  and  organizational  behaviors.  Within  this  focus  there  is  also  the  need  to  address  the  effect  of 
societal  groups:  cultural  factors,  religious  background,  and  individual  ethnicity  as  well  as  to  develop 
quantitative  and  qualitative  methods  for  the  representations  of  the  groups  and  their  responses  to  a  variety 
of  situations.  Another  major  focus  for  the  ET.V  is  to  determine  whether  the  group  is  a  set  of  individuals 
that  come  together  or  is  it  a  group  that  is  a  unit  that  can  possibly  decompose.  A  third  focus  is  the 
identification  of  experiments  based  upon  identified  broad  areas  of  need,  which  include  the  need  to  identify 
target  crowds  types  for  modeling;  the  response  of  different  cultures  to  information  warfare  activities; 
and  the  need  to  identify  team  situational  awareness.  Another  crucial  goal  for  IT.V  is  to  identify  applications 
that  have  high  relevance  to  military  missions  and  need,  such  as  a  C4I  application  area,  negations,  or  an 
application  area  that  supports  special  operations  forces  needs. 

A.2.3  Schedule  and  Meetings  Focus 

To  accomplish  the  taskings  and  produce  its  required  report  as  set  forth  by  the  SAS  Panel  within  the  time 
allotted,  the  ET.V  pursued  an  aggressive  schedule  that  allocated  a  total  of  one  year  for  the  complete  effort. 
To  achieve  the  schedule  and  organize  the  work,  the  SAS-ET.V  program  of  work  and  work  schedule  was 
broken  into  three  main  meetings  over  the  course  of  the  year.  The  first  meeting  was  held  in  October  2002  at 
the  Air  Force  Agency  for  Modeling  and  Simulation  in  Orlando,  FL.  The  focus  of  the  first  meeting  was  to 
determine  the  interest  of  the  country  participants  in  the  concept  of  a  virtual  institute  for  HBR  and  their 
ability  to  participate.  Each  country  representative  presented  a  statement  of  their  relevant  research  efforts 
and  their  particular  interest  in  the  SAS-ET.V  effort.  These  country  statements  included  discussions  of  the 
representative’s  background,  organization,  and  potential  contribution  to  the  SAS-ET.V  work  and  a  future 
HBR  Virtual  Institute  Technical  Team.  During  the  working  session,  SAS-ET.V  discussed  a  draft  HBR 
virtual  institute  Terms  of  Reference  and  then  went  directly  to  work  defining  the  Technical  Area  Plan 
(TAP)  required  to  define  and  complete  this  NATO  SAS-ET.V  effort.  The  final  discussions  during  the 
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first  SAS-ET.V  meeting  concentrated  on  the  focus  and  content  of  the  final  work  product  for  the  team, 
an  advisory  report  to  the  NATO  SAS  Panel  and  RTO.  The  Technology  Area  Plan  (TAP)  for  the 
SAS-ET.V  was  adopted  by  the  team  during  its  first  meeting. 

The  mid-term  meeting  for  SAS-ET.V  was  held  in  May  2003  in  conjunction  with  the  12th  Behavior 
Representation  for  Modeling  and  Simulation  (BRIMS,  formerly  the  Computer  Generated  Forces  and 
Behavioral  Representation  Conference).  This  meeting  focused  on  defining  the  structure  and  organization 
of  the  HBR  Virtual  Institute  and  the  Vi’s  interactions  and  operational  requirements.  The  team  also 
discussed  the  necessary  cooperation  and  coordination  necessary  with  other  RTO  Panels,  namely  the  HFM 
Panel,  the  1ST  Panel,  and  the  NMSG.  Each  team  member  was  tasked  with  identifying  and  gaining  support 
of  at  least  one,  preferably  two,  program(s)  within  their  representative  countries  that  would  be  candidate 
programs  for  inclusion  in  the  prototype  HBR  VI. 

The  final  meeting  of  SAS-ET.V  was  held  in  September  2003  at  NC3A  in  The  Hague,  Netherlands.  The  final 
meeting  allowed  the  team  to  finalize  its  efforts  in  documenting  the  requirements,  operation,  and  future  of  an 
HBR  VI.  The  final  team  meeting  also  discussed  the  recommendations  to  the  SAS  Panel  and  RTO  on  the 
establishment  of  a  RTO  Task  Group  (RTG)  to  begin  the  implementation  of  the  HBR  VI  development  plan 
defined  in  the  SAS-ET.V  advisory  report  and  reported  on  the  individual  country  project  interested  in 
participating  in  the  HBR  VI  prototype. 


A.3  APPROACH  FOR  A  PROTOTYPE  VIRTUAL  INSTITUTE 

A. 3.1  Requirements  Overview 

The  work  group  identified  a  number  of  requirements  for  the  HBR  virtual  institute.  The  number  of 
requirements  identified  to  date  greatly  exceeds  the  space  available  for  the  paper,  some  of  the  more  important 
requirements  identified  to  date  are  the  following: 

1)  Develop  a  virtual  institute  technology  development  strategy; 

2)  Develop  virtual  institute  metrics  and  success  criteria; 

3)  Develop  a  virtual  institute  research  plan  and  strategy; 

4)  Determine  the  key  steps  for  developing  the  virtual  institute  and  the  key  technologies  needed  by  the 
virtual  institute  to  conduct  initial  experiments; 

5)  Develop  a  methodology  to  assess  technical  benefits  of  virtual  institute-based  experiments  and  tests 
as  compared  to  conducting  equivalent  experiments  and  tests  in  one  location; 

6)  Develop  descriptions  of  partitions  of  work  between  national  participants  based  on  capabilities  and 
national  interests; 

7)  Develop  guidelines  for  technology  sharing  among  virtual  institute  participants; 

8)  Identify  and  prioritize  experiments  and  tests  that  are  of  common  interest  to  the  participating 
countries; 

9)  Develop  ontologies  for  human  behaviors; 

10)  Develop  and  employ  a  methodology  that  can  be  used  to  organize  a  human  behavior  model  library 
for  each  country  and  for  all  countries  as  well  as  enable  the  development  of  common  human 
behavior  models  and  model  components; 

11)  Develop  a  virtual  institute  human  behavior  model  experiment  plan; 

12)  Determine  common  behaviors  and  country  specific  behaviors  via  experiments; 
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13)  Determine  commander  behavior  modeling  techniques;  and 

14)  Develop  a  sequence  of  experiments  and  tests  that  will  demonstrate  benefits  and  desired  level  of 
international  cooperation  for  virtual  institute  human  behavior  model  development. 

In  addition  to  the  above  requirements,  the  HBR  virtual  institute  should  provide  other  capabilities  such  as 
serving  as  a  clearing  house  for  research  results,  providing  a  library  of  research  results,  ongoing  academic 
research,  and  experiments,  insuring  that  HBR  studies  are  coordinated  between  nations,  and  serving  as  a 
testbed  host  for  distributed  experiments.  In  our  view,  a  prime  function  of  the  virtual  institute  would  be  the 
distribution  of  human  performance  models  to  organizations  that  need  them.  The  institute  should  also  serve 
to  foster  cross-national  academic  research  support  for  HBR  model  development  and  support  validation  of 
models.  Another  important  contribution  that  the  institute  can  make  is  the  development/promulgation  of 
standards  or  measures  for  human  behavior  models,  developing  extensible  standards  for  describing/ 
defining  human  behavior  models,  and  determining  when  the  conditions/circumstances  where  each  model 
applies.  Another  key  issue  that  the  institute  should  address  is  the  development  of  composable  HBR  data 
models  and  development  of  the  means  for  storing  data  in  a  standard  manner  so  that  it  can  be  reused  across 
experiments.  In  addition  to  these  prime  issues,  there  are  several  secondary  issues  that  the  institute  should 
address  including  support  for  collaboration  and  virtual  meeting  support,  support  for  distance  learning, 
and  providing/supporting  meetings/open  forums  at  CGF/BR,  ICCM,  and  other  conferences  in  order  to 
engage  the  research  and  military  application  communities  and  make  them  aware  of  the  institute,  its  abilities, 
accomplishments,  and  community  needs. 

S  AS -ET.V  determined  early  in  its  effort  that  the  concept  to  be  pursued  was  that  of  a  virtual  institute  as 
opposed  to  a  virtual  laboratory.  The  concept  description  in  terms  of  an  institute  carries  a  broader  connotation 
whereas  a  laboratory  is  connotes  a  narrower  focus  to  the  activity.  In  light  of  the  broader  focus  of  the  virtual 
institute,  an  experimental  testbed  is  a  necessary  component  of  the  virtual  institute.  In  addition,  the  institute 
would  also  have  a  number  of  other  components,  such  as  library  and  information  and  modeling  clearing 
house.  The  institute  would  also  provide  training,  an  electronic  library,  a  news  center,  and  be  a  host  for 
exercises  and  technology  demonstrations.  The  institute  would  focus  on  addressing  new  areas  of  need  in 
human  behavior  modeling  for  NATO,  such  as  crowd  modeling  and  joint  training.  The  main  structure  of  the 
institute,  such  as  administration,  technical  support,  management,  and  oversight  would  be  relatively  fixed 
whereas  specific  experimental  topics  and  country  participants  would  vary.  This  concept  for  operations  and 
organization  led  to  a  flexible  organizational  structure  for  the  virtual  institute,  as  depicted  in  Figure  A-l. 
Some  portions  of  the  institute  continually  exist,  such  as  the  operational  executive,  the  administration 
function,  and  the  Multi-national  Control  Board.  These  components  of  the  organization  provide  guidance  and 
oversight  to  experimental  teams,  aid  in  the  formation  of  virtual  teams,  security,  and  aid  in  coordination 
between  teams.  The  technical  organization  is  another  permanent  part  of  the  virtual  institute;  its  purpose  is  to 
provide  ongoing  technical  services  to  all  teams,  aid  in  interchange  between  teams,  aid  in  making  the 
technical  connectivity  needed  by  the  virtual  teams  in  order  for  them  to  exist,  and  also  provide  data  storage, 
access,  protection,  data  mining,  and  security  and  privacy  services  for  the  institute’s  data.  The  technical 
organization  will  assist  the  virtual  experimental  teams  in  managing  and  conducting  their  experiments  and  in 
placing  their  raw  and  analyzed  data  into  permanent  storage.  These  support  organization  exist  to  aid  the 
virtual  HBR  experimental  teams  in  executing  their  experiments  and  research.  The  experimental  teams  would 
have  representatives  from  any  interested  member  nation.  The  experimental  team  would  have  a  well-defined 
research  objective,  such  as  the  effect  of  organization  on  HBR,  and  when  that  objective  is  achieved  that  team 
can  disband.  Intriguingly,  the  experimental  team  can  gain  or  lose  members  at  any  time  and  the  team  need 
never  meet  in  order  for  it  to  conduct  its  research;  it  is  the  responsibility  of  the  technical  organization  to  insure 
that  the  distance  between  members  does  not  interfere  with  their  progress  on  their  research. 
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Figure  A-1 :  Virtual  Institute  Definition  and  Organization. 

Based  on  the  described  requirements  above,  the  team  constructed  a  concept  diagram  of  the  operational  and 
process  elements  necessary  for  the  virtual  institute.  This  VI  operational  concept  diagram  is  shown  in 
Figure  A-2  -  Figure  A-5.  The  key  element  of  operation  for  the  virtual  institute  is  the  experimental  team, 
therefore,  all  of  the  virtual  institute’s  main  processes  are  intended  to  help  the  experimental  teams  achieve 
their  objectives.  Once  a  HBR  need  is  identified,  a  team  is  formed  and  the  team  begins  its  activities  by 
identifying  available  resources  and  models,  developing  an  experimental  plan  to  expand  the  available  HBR, 
and  evaluating  the  end-product  HBR.  In  general,  processing  proceeds  from  identification  of  national 
capabilities  and  needs  to  experimental  definition  and  resource  identification  and  development  to 
experimentation,  to  evaluation,  to  feedback.  Figure  A-2  shows  the  part  of  the  process  related  to  cultural 
modeling.  This  portion  of  the  process  calls  for  the  identification  of  national  capabilities  in  HBR  that  are 
relevant  to  an  experimental  team.  Once  the  national  capabilities  are  identified,  ontologies  based  on  the 
national  capabilities  are  formed  and  combined  into  a  single  common  ontology  for  the  experimental  team. 
In  parallel,  national  HBR  libraries  are  identified;  these  libraries  are  combined  by  the  technical  organization 
into  a  virtual  common  HBR  library.  In  addition  to  identifying  the  national  capabilities,  culture  specific 
organizational  behavior  models  are  identified  and  both  the  national  capabilities  and  organizational 
behavior  models  are  represented  commonly  possible  using,  for  example,  the  Unified  Modeling  Language 
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(UML).  At  this  point,  an  initial  common  HBR  for  the  experimental  team  has  been  formed.  In  parallel  with 
this  effort,  a  research  plan  and  strategy  to  verify  and  validate  the  model  and  to  extend  the  model  to  meet 
specific  needs  is  undertaken.  This  items  are  presented  to  the  virtual  institute  for  examination  and  a 
determination  whether  the  experimental  team  can  proceed. 


Figure  A-2:  HBR  Virtual  Institute  Model  Development  Process. 


Figure  A-3  contains  our  best  estimate  of  the  process  that  the  virtual  institute  should  employ  for 
experimental  planning.  As  with  model  development,  the  process  begins  with  the  identification  of  available 
national  capabilities  and  guidelines  for  sharing  data  between  various  members  of  the  experimental  team 
(some  new  guidelines  may  be  needed  from  time  to  time  as  conditions  warrant).  In  addition  to  determining 
the  guidelines  for  sharing,  the  cost  share  for  each  member  is  determined.  Finally,  an  assessment  of  key 
technologies  is  completed  so  that  the  team  will  know  which  technologies  can  be  employed  to  achieve  their 
research  objectives.  With  these  four  items  in  hand,  a  research  plan  and  strategy  is  developed  and  then  an 
experimental  plan  along  with  experimental  success  criteria  are  developed  to  support  the  research  plan. 
In  parallel  with  this  effort,  the  virtual  institute  support  structure  conducts  an  examination  of  the  state  of  the 
technological  art  needed  by  the  experimental  team,  assesses  technical  requirements  for  the  team,  and 
develops  some  technology  as  required.  The  key  technologies,  research  plan,  experimental  plan,  virtual 
institute  technology  assessment,  and  information  from  related  assessments  in  the  model  development 
process  are  then  combined  and  evaluated  by  the  virtual  institute.  If  the  modeling  effort  and  experimental 
plan  warrant  and  the  technology  exists  to  conduct  the  experiment,  the  virtual  institute  allows  the 
experimental  team  to  move  on  to  the  next  phase  of  the  process,  depicted  in  Figure  A-4. 


A- 10 


RT  O-TR-S  AS-053 


NATO 

OTAN 


ANNEX  A  -  ET.V  FINAL  REPORT 


Figure  A-3:  HBR  Virtual  Institute  Experimental  Planning  and  Support  Process. 
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Figure  A-4:  HBR  Virtual  Institute  Experimentation  and  Evaluation  Process  Flow. 

The  experimentation  and  evaluation  process  begins  with  the  model  data,  experimental  plan,  research 
strategy,  and  technological  underpinnings  identified  by  the  virtual  institute.  The  first  step  is  to  conduct  the 
initial  experiments  using  virtual  participants  as  necessary  and  available.  The  results  of  the  experiments  are 
evaluated  and  analyzed  using  previously  determined  evaluation  criteria  and  experiment  success  criteria. 
As  part  of  the  effort  to  demonstrate  the  utility  of  the  virtual  institute,  the  quality  of  the  results  obtained  using 
a  virtual  experimental  team  would  be  compared  with  a  “real”  team  and  the  cost  of  the  virtual  and  real 
approaches  would  be  compared.  As  in  all  good  scientific  endeavors,  we  plan  to  have  feedback  from  the 
results  of  the  experiments  and  analysis  into  the  earlier  portions  of  the  process  in  order  to  improve  them. 
For  the  sake  of  clarity,  these  feedback  arrows  were  omitted  from  the  figures  but,  in  our  view,  feedback  can 
arise  from  any  point  in  the  process  and  go  to  any  point  in  the  process.  Figure  A-5  shows  the  complete 
process,  with  duplicate  steps  removed  so  that  the  process  is  indicated  as  clearly  and  cleanly  as  possible. 
One  of  the  first  activities  for  the  follow-on  organization  should  be  to  test  and  evaluate  this  process  and 
determine  where  it  can  be  improved. 


A- 12 


RT  O-TR-S  AS-053 


RTO-TR-SAS-053 


A- 13 


ANNEX  A  -  ET.V  FINAL  REPORT 


To  conclude,  in  the  next  section  we  present  the  recommended  way  forward  for  the  implementation  of  a 
Virtual  Institute  for  HBR  in  NATO. 


A.4  WAY  FORWARD  FROM  ET.V 

Based  upon  the  meeting  results  and  investigations  conducted  by  the  ET.V  team,  the  SAS-ET.V  team 
recommends  that  a  RTO  Task  Group  (RTG)  be  established  to  develop  a  demonstration  Virtual  Institute 
and  that  the  Virtual  Institute  should  produce  a  Research  Plan  for  Team,  Group,  Organizational,  and 
Cultural  Modeling  Research. 

The  ET.V  has  developed  a  preliminary  Technical  Area  Plan  (TAP),  Draft  Terms  of  Reference  (TOR), 
and  Program  of  Work  (POW)  for  follow-on  RTG  to  proceed  directly  into  Prototype  Virtual  Institute 
implementation. 

Based  upon  work  done  to  date,  SAS-ET.V  has  additional  Recommendations  for  Virtual  Institute  use  and 
administration.  These  recommendations  follow. 

SAS-ET.V  recommends  that  one  of  the  first  tasks  that  the  RTG  should  undertake  is  drafting  a  Memorandum 
of  Understanding  (MOU)  to  address  intellectual  property  rights,  government  interests,  funding,  cost  sharing, 
results  sharing,  handling  of  security  and  classification  issues,  management,  commercial  support  and  other 
issues  to  be  addressed  as  part  of  international  agreements.  The  MOU  is  an  important  first  step  in  the 
development  of  the  virtual  institute  concept  because  it  will  lay  the  groundwork  for  the  scope  of  the  virtual 
institute  and  insure  the  protection  of  the  technologies  made  available  to  the  virtual  institute  by  the 
participating  countries. 

Based  on  our  experience  and  research  to  date,  the  SAS  ET.V  makes  the  following  three  recommendations 
that  should  be  undertaken  to  assemble,  test,  and  validate  a  Prototype  Virtual  Institute  and  thereby 
demonstrate  its  utility  and  also  test  the  process  flow  and  organizational  structure.  The  prototype  would  be 
able  to  handle  one  project  at  a  time,  unlike  multiple  parallel  projects  in  the  desired  virtual  institute  but  the 
prototype  would  serve  to  validate  the  concept  and  determine  the  utility  of  the  results  without  making  a  great 
funding  or  manpower  investment.  As  a  first  step,  we  recommend  the  development  of  a  common  ontology(s) 
and  common  interface  between  models  for  a  single  experiment  that  can  be  conducted  to  demonstrate  the 
value  of  the  virtual  institute  concept  and  also  show  that  valid  human  behavior  results  can  be  obtained  using  a 
virtual  institute.  This  suggestion  closely  mirrors  the  virtual  institute  process  flow  presented  above,  but  differs 
in  the  fact  that  only  one  program  would  be  executed  and  so  there  would  be  no  need  for  cross-project 
communication  or  collaboration.  The  second  step,  much  as  in  the  proposed  process  flow,  would  be  the 
development  of  use  cases  in  order  to  demonstrate  the  utility  of  the  virtual  institute  concept.  These  use  cases 
would  be  derived  from  operational  needs  and  the  operational  environment  and  would  be  written  to  address 
an  operational  need  via  experimentation  using  the  virtual  institute.  Here  again,  we  would  concentrate  on  one 
project  and  be  sure  that  it  is  a  project  suitable  to  the  institute’s  abilities  and  can  serve  as  a  useful 
demonstration  of  its  capabilities.  The  third  step  would  be  to  develop  a  series  of  sequential  projects,  derived 
from  these  use  cases  that  can  be  used  to  demonstrate  and  evaluate  the  complete  virtual  institute  process  and 
organizational  structure  without  having  to  deal  with  the  complexities  of  multiple  projects  or  multiple  HBR 
objectives  being  pursued  within  the  institute  simultaneously.  When  these  demonstration  projects  are 
successful,  a  decision  can  be  made  concerning  whether  to  develop  and  deploy  a  complete  virtual  institute. 

An  additional  recommendation  of  SAS-ET.V  is  that  the  RTG  improve  cooperation  between  the  HFM,  1ST, 
and  SAS  Panels,  the  NMSG,  and  NC3A  to  support  the  RTG  in  its  development  and  demonstration  of  a 
broad-scale  NATO  virtual  institute  capability.  To  achieve  this  goal,  SAS-ET.V  recommends  cooperation  and 
coordination  with  other  interested  (and  necessary)  Panels  to  participate  in  the  RTG  for  the  VI.  In  our  view, 
HFM  interest  would  be  in  cooperation  for  the  HBR  technical  aspects,  1ST  interest  would  be  in  cooperation 
for  the  virtual  institute  infrastructure  aspects,  and  SAS  interest  would  be  in  cooperation  for  the  concept 
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definition,  integration,  and  exploration  aspects.  In  addition,  work  must  be  performed  in  conjunction  with 
SAS  for  operational  analyses  support  and  NMSG  interest  would  be  in  cooperative  use  of  many  of  the  HBR 
developments  in  the  M&S  arena. 

One  area  of  clear  importance  to  the  Virtual  Institute  that  remains  to  be  addressed  is  increased  participation  by 
the  member  countries  in  various  projects  -  we  are  still  in  need  of  more  proposed  projects  and  support  from  at 
least  one  defined  and  funded  project  to  be  included  as  a  test  participant  in  the  HBR  VI  prototype.  These 
projects,  proposed  and  existing,  from  each  participating  country  will  be  included  in  the  prototype  NATO  VI 
and  become  the  foundation  for  the  exploration  of  the  NATO  VI  concept  which  will  lead  to  a  recommended 
design  and  development  process  for  a  VI  for  HBR  and  for  virtual  institutes  for  the  other  six  Panels. 
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